Out-of-distribution Detection on CIFAR-100 (ID) vs TinyImageNet (OOD) (test)
99AUROCOurs
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OursBackbone=DenseNet2019.12 | 99 | 95.7 | 95.5 | — | — | |
| OursModel Architecture=DenseNet2019.12 | 99 | 95.7 | 95.5 | — | — | |
| OursArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 98.97 | 95.12 | 95.13 | — | — | |
| OursBackbone=ResNet2019.12 | 98.9 | 94.8 | 95 | — | — | |
| OursModel Architecture=ResNet2019.12 | 98.9 | 94.8 | 95 | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=OOD samples2018.07 | 98.2 | 90.9 | 93.3 | — | — | |
| MahalanobisBackbone=ResNet2018.07 | 98.2 | 90.9 | 93.3 | 98.2 | 98.2 | |
| MahalanobisArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 98.2 | 90.92 | 93.3 | — | — | |
| MahalanobisBackbone=ResNet2019.12 | 98.2 | 90.9 | 93.3 | — | — | |
| MahalanobisModel Architecture=ResNet2019.12 | 98.2 | 90.9 | 93.3 | — | — | |
| OursModel Architecture=ResNet2019.12 | 97.7 | 88.5 | 92.2 | — | — | |
| OursModel Architecture=DenseNet2019.12 | 97.7 | 89 | 92.5 | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 97.4 | 86.6 | 92.2 | — | — | |
| MahalanobisBackbone=DenseNet2018.07 | 97.4 | 86.6 | 92.2 | 97.6 | 97.2 | |
| MahalanobisBackbone=DenseNet2019.12 | 97.4 | 86.6 | 92.2 | — | — | |
| MahalanobisModel Architecture=DenseNet2019.12 | 97.4 | 86.6 | 92.2 | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 97 | 87.2 | 91.8 | — | — | |
| MahalanobisModel Architecture=ResNet2019.12 | 96.3 | 80.9 | 89.9 | — | — | |
| VDArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 95.3 | 83.7 | 89.7 | — | — | |
| MahalanobisModel Architecture=DenseNet2019.12 | 88.8 | 60.1 | 81.6 | — | — | |
| ODINModel Architecture=DenseNet2019.12 | 88.3 | 51 | 80.2 | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 87.9 | 70.3 | 84.6 | — | — | |
| ODINBackbone=ResNet, Validation Setup=OOD samples2018.07 | 87.6 | 49.2 | 80.1 | — | — | |
| ODINBackbone=ResNet2018.07 | 87.6 | 49.2 | 80.1 | 87.1 | 87.4 | |
| ODINArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 87.6 | 36.1 | 80.1 | — | — | |
| ODINBackbone=ResNet2019.12 | 87.6 | 36.1 | 80.1 | — | — | |
| ODINModel Architecture=ResNet2019.12 | 87.6 | 36.1 | 80.1 | — | — | |
| ODINModel Architecture=ResNet2019.12 | 85.4 | 44.3 | 78.3 | — | — | |
| ODINBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 85.3 | 43.2 | 77.2 | — | — | |
| ODINBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 85.2 | 42.6 | 77 | — | — | |
| ODINBackbone=DenseNet2018.07 | 85.2 | 42.6 | 77 | 85.6 | 84.5 | |
| ODINBackbone=DenseNet2019.12 | 85.2 | 42.6 | 77 | — | — | |
| ODINModel Architecture=DenseNet2019.12 | 85.2 | 42.6 | 77 | — | — | |
| ODINBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 83.6 | 33.5 | 75.9 | — | — | |
| SemanticArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 83.1 | 37.6 | 75.6 | — | — | |
| BaselineModel Architecture=ResNet2019.12 | 79.7 | 24.3 | 72.5 | — | — | |
| L2EOOD score=Maximum Softmax Probability (MSP), Input Resolution=32x322024.08 | 79.1 | — | — | — | — | |
| DEOOD score=Maximum Softmax Probability (MSP), Input Resolution=32x322024.08 | 77.3 | — | — | — | — | |
| BaselineBackbone=ResNet, Validation Setup=OOD samples2018.07 | 77.2 | 20.4 | 70.8 | — | — | |
| BaselineBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 77.2 | 20.4 | 70.8 | — | — | |
| BaselineBackbone=ResNet2018.07 | 77.2 | 20.4 | 70.8 | 79.7 | 73.3 | |
| BaselineArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 77.2 | 20.4 | 70.8 | — | — | |
| BaselineBackbone=ResNet2019.12 | 77.2 | 20.4 | 70.8 | — | — | |
| BaselineModel Architecture=ResNet2019.12 | 77.2 | 20.4 | 70.8 | — | — | |
| HMCOOD score=Maximum Softmax Probability (MSP), Input Resolution=32x322024.08 | 77 | — | — | — | — | |
| CSGMCMCOOD score=Maximum Softmax Probability (MSP), Input Resolution=32x322024.08 | 76.5 | — | — | — | — | |
| BaselineModel Architecture=DenseNet2019.12 | 76.2 | 24.6 | 69 | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 71.7 | 17.6 | 65.7 | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 71.7 | 17.6 | 65.7 | — | — | |
| BaselineBackbone=DenseNet2018.07 | 71.7 | 17.6 | 65.7 | 74.2 | 69 | |
| BaselineBackbone=DenseNet2019.12 | 71.7 | 17.6 | 65.7 | — | — | |
| BaselineModel Architecture=DenseNet2019.12 | 71.7 | 17.6 | 65.7 | — | — |