Out-of-distribution Detection on SVHN TinyImageNet in-distribution out-of-distribution (test)
99.9AUROCMahalanobis distance-based score
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 99.9 | 99.9 | 98.9 | — | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=OOD samples2018.07 | 99.9 | 99.9 | 99.1 | — | — | — | |
| MahalanobisBackbone=DenseNet2018.07 | 99.9 | 99.9 | 98.9 | 99.9 | 99.6 | — | |
| MahalanobisBackbone=ResNet2018.07 | 99.9 | 99.9 | 99.1 | 99.9 | 99.1 | — | |
| MahalanobisBackbone=DenseNet2019.12 | 99.9 | 99.9 | 98.9 | — | — | — | |
| MahalanobisBackbone=ResNet2019.12 | 99.9 | 99.9 | 99.1 | — | — | — | |
| MahalanobisModel Architecture=DenseNet2019.12 | 99.9 | 99.9 | 98.9 | — | — | — | |
| MahalanobisModel Architecture=ResNet2019.12 | 99.9 | 99.9 | 99.1 | — | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 99.8 | 99.9 | 98.9 | — | — | — | |
| OursBackbone=DenseNet2019.12 | 99.7 | 99.1 | 97.9 | — | — | — | |
| OursBackbone=ResNet2019.12 | 99.7 | 99.3 | 97.9 | — | — | — | |
| OursModel Architecture=DenseNet2019.12 | 99.7 | 99.1 | 97.9 | — | — | — | |
| OursModel Architecture=ResNet2019.12 | 99.7 | 99.3 | 97.9 | — | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 99.3 | 99.2 | 98.8 | — | — | — | |
| One-vs-All Oracle2022.01 | 95.75 | — | — | 59.69 | 99.59 | 10.3 | |
| ODINBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 95.1 | 84.1 | 90.4 | — | — | — | |
| ODINBackbone=DenseNet2018.07 | 95.1 | 84.1 | 90.4 | 97.1 | 91.4 | — | |
| ODINBackbone=DenseNet2019.12 | 95.1 | 84.1 | 90.4 | — | — | — | |
| ODINModel Architecture=DenseNet2019.12 | 95.1 | 84.1 | 90.4 | — | — | — | |
| UQGANMC-Dropout=true2022.01 | 94.96 | — | — | 59.76 | 99.51 | 13.72 | |
| BaselineBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 94.8 | 79.8 | 90.2 | — | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 94.8 | 79.8 | 90.2 | — | — | — | |
| ODINBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 94.8 | 79.8 | 90.2 | — | — | — | |
| BaselineBackbone=DenseNet2018.07 | 94.8 | 79.8 | 90.2 | 97.2 | 88.4 | — | |
| BaselineBackbone=DenseNet2019.12 | 94.8 | 79.8 | 90.2 | — | — | — | |
| BaselineModel Architecture=DenseNet2019.12 | 94.8 | 79.8 | 90.2 | — | — | — | |
| BaselineBackbone=ResNet, Validation Setup=OOD samples2018.07 | 93.5 | 79 | 90.4 | — | — | — | |
| BaselineBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 93.5 | 79 | 90.4 | — | — | — | |
| BaselineBackbone=ResNet2018.07 | 93.5 | 79 | 90.4 | 95.7 | 86.2 | — | |
| BaselineBackbone=ResNet2019.12 | 93.5 | 79 | 90.4 | — | — | — | |
| BaselineModel Architecture=ResNet2019.12 | 93.5 | 79 | 90.4 | — | — | — | |
| ODINBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 92.9 | 80.5 | 90.1 | — | — | — | |
| ODINBackbone=ResNet, Validation Setup=OOD samples2018.07 | 92 | 82.1 | 89.4 | — | — | — | |
| ODINBackbone=ResNet2018.07 | 92 | 82.1 | 89.4 | 93.9 | 88.1 | — | |
| ODINBackbone=ResNet2019.12 | 92 | 82 | 89.4 | — | — | — | |
| ODINModel Architecture=ResNet2019.12 | 92 | 82 | 89.4 | — | — | — | |
| Entropy Oracle2022.01 | 85.43 | — | — | 41.95 | 98.27 | 48.74 | |
| GEN2022.01 | 84.65 | — | — | 36.86 | 98.28 | 39.67 | |
| UQGANMC-Dropout=false2022.01 | 79.25 | — | — | 26.35 | 97.66 | 47.22 | |
| Bayes-by-Backprop2022.01 | 68.05 | — | — | 21.24 | 95.23 | 81.7 | |
| Deep-Ensembles2022.01 | 67.76 | — | — | 30.85 | 93.93 | 93.79 | |
| MC-Dropout2022.01 | 63.35 | — | — | 27.11 | 92.78 | 95.86 | |
| Entropy2022.01 | 62.44 | — | — | 21.9 | 93.16 | 93.79 | |
| Max. Softmax2022.01 | 61.53 | — | — | 21.07 | 93.29 | 92.51 | |
| Confident Classifier2022.01 | 59.99 | — | — | 20.58 | 92.66 | 94.49 | |
| One-vs-All Baseline2022.01 | 55.19 | — | — | 17.97 | 90.97 | 97.32 |