Out-of-distribution Detection on CIFAR-100 (ID) vs LSUN (OOD) (test)
99.3AUROCOurs
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OursBackbone=DenseNet2019.12 | 99.3 | 97.2 | 96.4 | — | — | — | — | |
| OursArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 99.28 | 97.14 | 96.19 | — | — | — | — | |
| OursBackbone=ResNet2019.12 | 99.2 | 96.6 | 96.7 | — | — | — | — | |
| Deep Abstaining ClassifierBackbone=ResNet 342021.05 | 98.45 | — | — | — | — | 7.14 | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=OOD samples2018.07 | 98.2 | 90.9 | 93.5 | — | — | — | — | |
| MahalanobisBackbone=ResNet2018.07 | 98.2 | 90.9 | 93.5 | 98.4 | 97.8 | — | — | |
| MahalanobisArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 98.2 | 90.89 | 93.5 | — | — | — | — | |
| MahalanobisBackbone=ResNet2019.12 | 98.2 | 90.9 | 93.5 | — | — | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 98 | 91.4 | 93.9 | — | — | — | — | |
| MahalanobisBackbone=DenseNet2018.07 | 98 | 91.4 | 93.9 | 98.2 | 97.5 | — | — | |
| MahalanobisBackbone=DenseNet2019.12 | 98 | 91.4 | 93.9 | — | — | — | — | |
| Mahalanobis distance-based scoreBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 97.9 | 91.4 | 93.8 | — | — | — | — | |
| Deep Abstaining ClassifierBackbone=Wide ResNet 28x102021.05 | 97.89 | — | — | — | — | 9.23 | — | |
| Deep Abstaining ClassifierBackbone=Wide ResNet 40x22021.05 | 97.67 | — | — | — | — | 9.27 | — | |
| Ensemble of Leave-out ClassifiersBackbone=Wide ResNet 28x102021.05 | 96.77 | — | — | — | — | 16.53 | — | |
| VDArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 95.9 | 85.5 | 90.4 | — | — | — | — | |
| NUQBackbone=ResNet-50, Single-pass capability=true, Density estimate=GMM2022.02 | 92.3 | — | — | — | — | — | — | |
| DDUBackbone=ResNet-50, Single-pass capability=true2022.02 | 92.1 | — | — | — | — | — | — | |
| DUQBackbone=ResNet-50, Single-pass capability=true2022.02 | 90.8 | — | — | — | — | — | — | |
| REGradBackbone=ViT2024.04 | 88.38 | — | — | — | — | — | 85.1 | |
| EnsembleBackbone=ResNet-50, Single-pass capability=false, Number of models=52022.02 | 86.5 | — | — | — | — | — | — | |
| ODINBackbone=Wide ResNet 28x102021.05 | 86 | — | — | — | — | 56.5 | — | |
| REGrad*layer-selective=true, perturbation=true2024.04 | 85.74 | — | — | — | — | — | 74.39 | |
| ODINBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 85.7 | 42.1 | 77.3 | — | — | — | — | |
| Perturb θBackbone=ViT2024.04 | 85.7 | — | — | — | — | — | 77.74 | |
| ODINBackbone=ResNet, Validation Setup=OOD samples2018.07 | 85.6 | 45.6 | 78.3 | — | — | — | — | |
| ODINBackbone=ResNet2018.07 | 85.6 | 45.6 | 78.3 | 84.5 | 85.7 | — | — | |
| ODINArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 85.6 | 23.2 | 78.3 | — | — | — | — | |
| ODINBackbone=ResNet2019.12 | 85.6 | 23.2 | 78.3 | — | — | — | — | |
| ExGradBackbone=ViT2024.04 | 85.55 | — | — | — | — | — | 79.68 | |
| ODINBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 85.5 | 41.2 | 77.1 | — | — | — | — | |
| ODINBackbone=DenseNet2018.07 | 85.5 | 41.2 | 77.1 | 86.4 | 84.2 | — | — | |
| ODINBackbone=DenseNet2019.12 | 85.5 | 41.2 | 77.1 | — | — | — | — | |
| TTABackbone=ResNet-50, Single-pass capability=false2022.02 | 85 | — | — | — | — | — | — | |
| SNGPBackbone=ResNet-50, Single-pass capability=true2022.02 | 83.7 | — | — | — | — | — | — | |
| Outlier ExposureBackbone=Wide ResNet 40x22021.05 | 83.4 | — | — | — | — | 57.5 | — | |
| OpenMaxBackbone=ResNet 342021.05 | 83.08 | — | — | — | — | 30.21 | — | |
| EntropyBackbone=ResNet-50, Single-pass capability=true2022.02 | 83 | — | — | — | — | — | — | |
| EnergyBackbone=ResNet-50, Single-pass capability=true2022.02 | 82.7 | — | — | — | — | — | — | |
| Mahalanobis distance-based scoreBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 82.3 | 56.6 | 79.7 | — | — | — | — | |
| Deep Mahalanobis DetectorBackbone=ResNet 342021.05 | 82.3 | — | — | — | — | 43.4 | — | |
| ODINBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 81.9 | 31.6 | 74.6 | — | — | — | — | |
| MaxProbBackbone=ResNet-50, Single-pass capability=true2022.02 | 81.5 | — | — | — | — | — | — | |
| SemanticArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 79 | 20.5 | 57.8 | — | — | — | — | |
| Exgrad2024.04 | 76.93 | — | — | — | — | — | 71 | |
| DropoutBackbone=ResNet-50, Single-pass capability=false2022.02 | 76.8 | — | — | — | — | — | — | |
| LA2024.04 | 76.5 | — | — | — | — | — | 71.92 | |
| BaselineBackbone=ResNet, Validation Setup=OOD samples2018.07 | 75.8 | 18.8 | 69.9 | — | — | — | — | |
| BaselineBackbone=ResNet, Validation Setup=adversarial samples2018.07 | 75.8 | 18.8 | 69.9 | — | — | — | — | |
| BaselineBackbone=ResNet2018.07 | 75.8 | 18.8 | 69.9 | 77.6 | 72 | — | — | |
| BaselineArchitecture=ResNet, In-distribution Dataset=CIFAR-1002019.12 | 75.8 | 18.8 | 69.9 | — | — | — | — | |
| BaselineBackbone=ResNet2019.12 | 75.8 | 18.8 | 69.9 | — | — | — | — | |
| UNGradBackbone=ViT2024.04 | 72.08 | — | — | — | — | — | 60.19 | |
| BaselineBackbone=DenseNet, Validation Setup=OOD samples2018.07 | 70.8 | 16.7 | 64.9 | — | — | — | — | |
| BaselineBackbone=DenseNet, Validation Setup=adversarial samples2018.07 | 70.8 | 16.7 | 64.9 | — | — | — | — | |
| BaselineBackbone=DenseNet2018.07 | 70.8 | 16.7 | 64.9 | 74.1 | 67.9 | — | — | |
| BaselineBackbone=DenseNet2019.12 | 70.8 | 17.6 | 64.9 | — | — | — | — | |
| GradNormBackbone=ViT2024.04 | 63.76 | — | — | — | — | — | 57.2 | |
| Inserted Dropout2024.04 | 59.41 | — | — | — | — | — | 58.3 | |
| UNGrad2024.04 | 47.46 | — | — | — | — | — | 45.41 | |
| NEGrad2024.04 | 43.99 | — | — | — | — | — | 42.74 | |
| Entropy2024.04 | 42.28 | — | — | — | — | — | 46.07 | |
| ExGrad V Term2024.04 | 41.67 | — | — | — | — | — | 45.91 | |
| Perturb x2024.04 | 41.1 | — | — | — | — | — | 44.35 | |
| Perturb θ2024.04 | 39.42 | — | — | — | — | — | 42.57 | |
| MC-AA2024.04 | 31.61 | — | — | — | — | — | 39.4 | |
| GradNorm2024.04 | 27.53 | — | — | — | — | — | 36.97 |