Out-of-distribution Detection on SVHN (ID) vs LSUN (OOD) (test)
100TNR @ TPR 95%Mahalanobis distance-based score
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 100 | 99.9 | 99.3 | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 100 | 99.9 | 99.2 | — | — | |
| MahalanobisBackbone=DenseNet2018.07 | 100 | 99.9 | 99.3 | 99.9 | 99.7 | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=OOD samples2018.07 | 99.9 | 99.9 | 99.5 | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 99.9 | 99.9 | 99.5 | — | — | |
| MahalanobisBackbone=ResNet2018.07 | 99.9 | 99.9 | 99.5 | 99.9 | 99.1 | |
| MahalanobisBackbone=DenseNet2019.12 | 99.9 | 99.9 | 99.3 | — | — | |
| MahalanobisBackbone=ResNet2019.12 | 99.9 | 99.9 | 99.5 | — | — | |
| MahalanobisModel Architecture=DenseNet2019.12 | 99.9 | 99.9 | 99.3 | — | — | |
| MahalanobisModel Architecture=ResNet2019.12 | 99.9 | 99.9 | 99.5 | — | — | |
| OursBackbone=ResNet2019.12 | 99.6 | 99.8 | 98.5 | — | — | |
| OursModel Architecture=ResNet2019.12 | 99.6 | 99.8 | 98.5 | — | — | |
| OursBackbone=DenseNet2019.12 | 99.5 | 99.8 | 98.6 | — | — | |
| OursModel Architecture=DenseNet2019.12 | 99.5 | 99.8 | 98.6 | — | — | |
| ODINBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 81.1 | 94.5 | 89.2 | — | — | |
| ODINBackbone=DenseNet2018.07 | 81.1 | 94.5 | 89.2 | 97 | 90.5 | |
| ODINBackbone=DenseNet2019.12 | 81.1 | 94.5 | 89.2 | — | — | |
| ODINModel Architecture=DenseNet2019.12 | 81.1 | 94.5 | 89.2 | — | — | |
| ODINBackbone=ResNet, Validation Setup=OOD samples2018.07 | 77.3 | 89.4 | 87.2 | — | — | |
| ODINBackbone=ResNet2018.07 | 77.3 | 89.4 | 87.2 | 92.1 | 85.5 | |
| ODINBackbone=ResNet2019.12 | 77.3 | 89.4 | 87.2 | — | — | |
| ODINModel Architecture=ResNet2019.12 | 77.3 | 89.4 | 87.2 | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 77.1 | 94.1 | 89.1 | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 77.1 | 94.1 | 89.1 | — | — | |
| ODINBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 77.1 | 94.1 | 89.1 | — | — | |
| BaselineBackbone=DenseNet2018.07 | 77.1 | 94.1 | 89.1 | 97 | 87.4 | |
| BaselineBackbone=DenseNet2019.12 | 77.1 | 94.1 | 89.1 | — | — | |
| BaselineModel Architecture=DenseNet2019.12 | 77.1 | 94.1 | 89.1 | — | — | |
| ODINBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 76.3 | 90.7 | 88.2 | — | — | |
| BaselineBackbone=ResNet, Validation Setup=OOD samples2018.07 | 74.3 | 91.6 | 89 | — | — | |
| BaselineBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 74.3 | 91.6 | 89 | — | — | |
| BaselineBackbone=ResNet2018.07 | 74.3 | 91.6 | 89 | 94.2 | 84 | |
| BaselineBackbone=ResNet2019.12 | 74.3 | 91.6 | 89 | — | — | |
| BaselineModel Architecture=ResNet2019.12 | 74.3 | 91.6 | 89 | — | — |