Out-of-distribution Detection on CIFAR-10 ID vs LSUN OOD (test)
99.9AUROCOurs
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OursBackbone=ResNet2019.12 | 99.9 | 99.6 | 98.6 | — | — | — | — | |
| OursBackbone=DenseNet2019.12 | 99.9 | 99.5 | 98.6 | — | — | — | — | |
| OursBackbone=ResNet2019.12 | 99.89 | 99.85 | 98.66 | — | — | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=OOD samples2018.07 | 99.7 | 98.9 | 97.7 | — | — | — | — | |
| MahalanobisBackbone=ResNet2018.07 | 99.7 | 98.9 | 97.7 | 99.7 | 99.7 | — | — | |
| MahalanobisBackbone=ResNet2019.12 | 99.7 | 98.8 | 97.7 | — | — | — | — | |
| MahalanobisBackbone=ResNet2019.12 | 99.7 | 98.8 | 97.7 | — | — | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 99.5 | 98.1 | 97.2 | — | — | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 99.3 | 97.2 | 96.3 | — | — | — | — | |
| MahalanobisBackbone=DenseNet2018.07 | 99.3 | 97.2 | 96.3 | 99.3 | 99.1 | — | — | |
| MahalanobisBackbone=DenseNet2019.12 | 99.3 | 97.2 | 96.3 | — | — | — | — | |
| ODINBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 99.2 | 96.2 | 95.7 | — | — | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 99.2 | 97.2 | 96.2 | — | — | — | — | |
| ODINBackbone=DenseNet2018.07 | 99.2 | 96.2 | 95.7 | 99.3 | 99.2 | — | — | |
| ODINBackbone=DenseNet2019.12 | 99.2 | 96.2 | 95.7 | — | — | — | — | |
| AENIBScore=log Dir_0.05(y) + log N(z_n; 0, I)2023.03 | 99 | — | — | — | — | — | — | |
| D_alphadetector=DOCTOR, variant=alpha2021.06 | 98.6 | — | — | — | — | 5.4 | — | |
| ODINBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 98.5 | 92.9 | 94.3 | — | — | — | — | |
| VDBackbone=ResNet2019.12 | 98.3 | 92.3 | 94.1 | — | — | — | — | |
| D_betadetector=DOCTOR, variant=beta2021.06 | 98.2 | — | — | — | — | 6.9 | — | |
| ODIN_OODbaseline=ODIN2021.06 | 98.2 | — | — | — | — | 8.7 | — | |
| OursModel Architecture=ResNet2019.12 | 97.8 | 89.8 | 92.6 | — | — | — | — | |
| OursModel Architecture=DenseNet2019.12 | 97.5 | 88.4 | 92 | — | — | — | — | |
| MahalanobisModel Architecture=ResNet2019.12 | 96.7 | 81.3 | 90.5 | — | — | — | — | |
| VIBScore=log Dir_0.05(y)2023.03 | 96 | — | — | — | — | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 95.4 | 66.6 | 90.3 | — | — | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 95.4 | 66.6 | 90.3 | — | — | — | — | |
| BaselineBackbone=DenseNet2018.07 | 95.4 | 66.6 | 90.3 | 96.5 | 94.1 | — | — | |
| BaselineBackbone=DenseNet2019.12 | 95.4 | 66.6 | 90.3 | — | — | — | — | |
| Cross-entropyScore=log Dir_0.05(y)2023.03 | 95 | — | — | — | — | — | — | |
| AENIBScore=log Dir_0.05(y)2023.03 | 95 | — | — | — | — | — | — | |
| REGradBackbone=ViT2024.04 | 94.99 | — | — | — | — | — | 94.06 | |
| ODINBackbone=ResNet, Validation Setup=OOD samples2018.07 | 94.1 | 73.8 | 86.7 | — | — | — | — | |
| ODINBackbone=ResNet2018.07 | 94.1 | 73.8 | 86.7 | 94.2 | 94.3 | — | — | |
| ODINBackbone=ResNet2019.12 | 94.1 | 82.1 | 86.7 | — | — | — | — | |
| ODINBackbone=ResNet2019.12 | 94.1 | 82.1 | 86.7 | — | — | — | — | |
| Cross-entropyScore=max_y p(y|x)2023.03 | 94 | — | — | — | — | — | — | |
| VIBScore=max_y p(y|x)2023.03 | 94 | — | — | — | — | — | — | |
| ODINBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 93.7 | 70 | 85.8 | — | — | — | — | |
| ODINModel Architecture=DenseNet2019.12 | 93.6 | 70.6 | 86.4 | — | — | — | — | |
| Auxiliary Rotation Predictionaverage_of_runs=52019.06 | 93.2 | — | — | — | — | 28.7 | 71 | |
| SupConScore=max_y p(y|x)2023.03 | 93 | — | — | — | — | — | — | |
| Clipped HNNScoring function=softmax score2021.07 | 92.97 | — | — | — | — | 41.49 | 98.46 | |
| BaselineModel Architecture=DenseNet2019.12 | 92.9 | 51.8 | 86.9 | — | — | — | — | |
| Vanilla HNNScoring function=softmax score2021.07 | 92.67 | — | — | — | — | 41.06 | 98.42 | |
| BaselineModel Architecture=ResNet2019.12 | 91.9 | 48.6 | 86.3 | — | — | — | — | |
| ODINModel Architecture=ResNet2019.12 | 91.2 | 62 | 82.4 | — | — | — | — | |
| REGrad*layer-selective=true, perturbation=true2024.04 | 91.11 | — | — | — | — | — | 89.93 | |
| BaselineBackbone=ResNet, Validation Setup=OOD samples2018.07 | 91 | 45.4 | 85.3 | — | — | — | — | |
| BaselineBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 91 | 45.4 | 85.3 | — | — | — | — | |
| BaselineBackbone=ResNet2018.07 | 91 | 45.4 | 85.3 | 92.5 | 88.6 | — | — | |
| BaselineBackbone=ResNet2019.12 | 91 | 49.8 | 85.3 | — | — | — | — | |
| BaselineBackbone=ResNet2019.12 | 91 | 49.8 | 85.3 | — | — | — | — | |
| Perturb θBackbone=ViT2024.04 | 90.7 | — | — | — | — | — | 81.59 | |
| DPNBackbone=ResNet2019.12 | 90.2 | 42.6 | 79.5 | — | — | — | — | |
| ExGrad V Term2024.04 | 89.96 | — | — | — | — | — | 88.09 | |
| Entropy2024.04 | 89.84 | — | — | — | — | — | 87.72 | |
| ExGradBackbone=ViT2024.04 | 89.06 | — | — | — | — | — | 86.55 | |
| Maximum Softmax Probability (MSP)average_of_runs=52019.06 | 88.5 | — | — | — | — | 39.5 | 57.2 | |
| Perturb θ2024.04 | 88.33 | — | — | — | — | — | 82.99 | |
| Exgrad2024.04 | 88.23 | — | — | — | — | — | 82.26 | |
| Inserted Dropout2024.04 | 88.18 | — | — | — | — | — | 86.93 | |
| AENIBScore=max_y p(y|x)2023.03 | 88 | — | — | — | — | — | — | |
| LA2024.04 | 86.81 | — | — | — | — | — | 85.04 | |
| UNGradBackbone=ViT2024.04 | 84.83 | — | — | — | — | — | 76.61 | |
| MC-AA2024.04 | 82.96 | — | — | — | — | — | 71.79 | |
| Perturb x2024.04 | 82.89 | — | — | — | — | — | 72.35 | |
| MahalanobisModel Architecture=DenseNet2019.12 | 80.2 | 48.2 | 75.6 | — | — | — | — | |
| GradNormBackbone=ViT2024.04 | 72.84 | — | — | — | — | — | 71.04 | |
| NEGrad2024.04 | 66.96 | — | — | — | — | — | 58.51 | |
| ENNScoring function=softmax score2021.07 | 22.05 | — | — | — | — | 99.62 | 71.88 | |
| UNGrad2024.04 | 17.08 | — | — | — | — | — | 33.52 | |
| GradNorm2024.04 | 10.57 | — | — | — | — | — | 32.04 |