Multi-Class Out-of-Distribution Detection on CIFAR-10 (ID) vs Textures (OOD) (test)
0.89FPR@95AHGC
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| AHGCBackbone=ResNet-18, Optimizer=SGD2024.12 | 0.89 | 99.42 | — | 99.67 | 99.05 | 54.03 | 75.02 | 87.81 | 94.62 | |
| Auxiliary Rotation Predictionaverage_of_runs=52019.06 | 8.9 | 97.4 | 86.7 | — | — | — | — | — | — | |
| SconeBackbone=ResNet-18, Optimizer=SGD2024.12 | 18.26 | 97.3 | — | 97.88 | 93.11 | 21.08 | 40.92 | 68.43 | 88.29 | |
| FedRoD+FOOGDalpha=0.12024.10 | 19.46 | 93.39 | — | — | — | — | — | — | — | |
| UDGBackbone=ResNet-18, Optimizer=SGD2024.12 | 20.43 | 96.44 | — | 98.12 | 92.91 | 19.9 | 43.33 | 69.19 | 87.71 | |
| ConjNormBackbone=ResNet-18, Optimizer=SGD2024.12 | 23.78 | 95.59 | — | 97.4 | 95.88 | 18.42 | 45.29 | 70.96 | 89.73 | |
| FedSDWCalpha=0.12025.11 | 37.2 | 87.86 | — | — | — | — | — | — | — | |
| FOSTERalpha=0.12024.10 | 39.22 | 81.52 | — | — | — | — | — | — | — | |
| FedRoDalpha=0.12024.10 | 39.55 | 83.07 | — | — | — | — | — | — | — | |
| ODINBackbone=ResNet-18, Optimizer=SGD2024.12 | 42.52 | 84.06 | — | 86.01 | 80.73 | 0.02 | 0.18 | 3.71 | 40.14 | |
| FedTHEalpha=0.12024.10 | 42.95 | 85.56 | — | — | — | — | — | — | — | |
| Maximum Softmax Probability (MSP)average_of_runs=52019.06 | 45.4 | 87.9 | 56.2 | — | — | — | — | — | — | |
| Clipped HNNScoring function=softmax score2021.07 | 47.12 | 89.91 | 97.39 | — | — | — | — | — | — | |
| Vanilla HNNsScore=Energy score2021.07 | 49.7 | 90.66 | 97.98 | — | — | — | — | — | — | |
| OEBackbone=ResNet-18, Optimizer=SGD2024.12 | 51.17 | 89.56 | — | 93.79 | 81.88 | 6.58 | 11.8 | 27.99 | 71.13 | |
| FedICONalpha=0.12025.11 | 51.57 | 80.96 | — | — | — | — | — | — | — | |
| EBOBackbone=ResNet-18, Optimizer=SGD2024.12 | 52.11 | 80.7 | — | 83.34 | 75.2 | 0.01 | 0.13 | 2.79 | 31.96 | |
| FedICONalpha=0.12024.10 | 53.58 | 82.19 | — | — | — | — | — | — | — | |
| FedTHEalpha=0.12025.11 | 53.58 | 82.19 | — | — | — | — | — | — | — | |
| FOSTERalpha=0.12025.11 | 54.23 | 77.62 | — | — | — | — | — | — | — | |
| Clipped HNNsScore=Energy score2021.07 | 58.51 | 82.98 | 94.55 | — | — | — | — | — | — | |
| Vanilla HNNScoring function=softmax score2021.07 | 65.04 | 82.8 | 94.59 | — | — | — | — | — | — | |
| FedATOLalpha=0.12025.11 | 66.33 | 78.77 | — | — | — | — | — | — | — | |
| FedAvg+FOOGDalpha=0.12024.10 | 71.99 | 88.6 | — | — | — | — | — | — | — | |
| PerAdaalpha=0.12025.11 | 76.81 | 68.25 | — | — | — | — | — | — | — | |
| FedAvgalpha=0.12024.10 | 77.98 | 57.86 | — | — | — | — | — | — | — | |
| FedIIRalpha=0.12024.10 | 78.34 | 79.89 | — | — | — | — | — | — | — | |
| FedAvgalpha=0.12025.11 | 80.53 | 66.23 | — | — | — | — | — | — | — | |
| FedATOLalpha=0.12024.10 | 82.17 | 64.04 | — | — | — | — | — | — | — | |
| FedLNalpha=0.12024.10 | 83.68 | 64.01 | — | — | — | — | — | — | — | |
| MCDBackbone=ResNet-18, Optimizer=SGD2024.12 | 83.92 | 81.59 | — | 90.2 | 63.27 | 4.97 | 10.51 | 29.52 | 62.1 | |
| FedIIRalpha=0.12025.11 | 91.72 | 62.32 | — | — | — | — | — | — | — | |
| FedLNalpha=0.12025.11 | 93.9 | 71.99 | — | — | — | — | — | — | — | |
| ENNsScore=Energy score2021.07 | 95.93 | 35.02 | 74.87 | — | — | — | — | — | — | |
| ENNScoring function=softmax score2021.07 | 97.62 | 33.87 | 74.52 | — | — | — | — | — | — |