Out-of-Distribution Detection on ImageNet-O (AUROC/FPR95)
0.953AUROCViM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ViMModel=DeiT, Source=feat+logit2022.03 | 0.953 | 0.894 | |
| MahalanobisModel=ViT-B/16, Source=feat+label2022.03 | 0.9281 | 0.3695 | |
| ViMModel=ViT-B/16, Source=feat+logit2022.03 | 0.9255 | 0.3675 | |
| ReActModel=ViT-B/16, Source=feat+logit2022.03 | 0.9071 | 0.426 | |
| EnergyModel=ViT-B/16, Source=logit2022.03 | 0.9046 | 0.413 | |
| ODINModel=ViT-B/16, Source=prob+grad2022.03 | 0.8985 | 0.4415 | |
| MaxLogitModel=ViT-B/16, Source=logit2022.03 | 0.8985 | 0.4415 | |
| ViMModel=Swin, Source=feat+logit2022.03 | 0.8878 | 0.592 | |
| ResidualModel=ViT-B/16, Source=feat2022.03 | 0.8823 | 0.4785 | |
| ResidualModel=Swin, Source=feat2022.03 | 0.8668 | 0.6855 | |
| MahalanobisModel=Swin, Source=feat+label2022.03 | 0.8546 | 0.7355 | |
| KL MatchingModel=ViT-B/16, Source=prob2022.03 | 0.8412 | 0.557 | |
| ReActModel=Swin, Source=feat2022.03 | 0.8409 | 0.445 | |
| KNNBackbone=ResNet-502026.02 | 0.84 | 0.649 | |
| ViMModel=BiT-S-R101x1, Source=feat+logit2022.03 | 0.8387 | 0.615 | |
| NUQBackbone=ResNet-50, Single-pass uncertainty estimation=true, Density estimator=KDE2022.02 | 0.824 | — | |
| EnergyModel=Swin, Source=logit2022.03 | 0.8229 | 0.457 | |
| WDiscOODBackbone=ResNet-502026.02 | 0.822 | 0.641 | |
| KL MatchingModel=Swin, Source=prob2022.03 | 0.8191 | 0.6735 | |
| MSPModel=ViT-B/16, Source=prob2022.03 | 0.8186 | 0.6485 | |
| ResidualModel=BiT-S-R101x1, Source=feat2022.03 | 0.8157 | 0.655 | |
| MaxLogitModel=Swin, Source=logit2022.03 | 0.8128 | 0.515 | |
| ResidualModel=Res50d, Source=feat2022.03 | 0.8115 | 0.7285 | |
| ViMModel=Res50d, Source=feat+logit2022.03 | 0.8102 | 0.748 | |
| ODINModel=Swin, Source=prob+grad2022.03 | 0.8062 | 0.5365 | |
| MahalanobisBackbone=ResNet-502026.02 | 0.806 | 0.671 | |
| MahalanobisModel=BiT-S-R101x1, Source=feat+label2022.03 | 0.8037 | 0.7005 | |
| MahalanobisModel=Res50d, Source=feat+label2022.03 | 0.8015 | 0.76 | |
| D-KNNBackbone=ResNet-502026.02 | 0.791 | 0.745 | |
| MSPModel=Swin, Source=prob2022.03 | 0.7897 | 0.637 | |
| ViMModel=RepVGG, Source=feat+logit2022.03 | 0.7693 | 0.7905 | |
| MahalanobisModel=RepVGG, Source=feat+label2022.03 | 0.7668 | 0.818 | |
| MahalanobisModel=DeiT, Source=feat+label2022.03 | 0.7595 | 0.9025 | |
| SNGPBackbone=ResNet-50, Single-pass uncertainty estimation=true2022.02 | 0.758 | — | |
| NecoBackbone=ResNet-502026.02 | 0.758 | 0.732 | |
| ResidualModel=RepVGG, Source=feat2022.03 | 0.7511 | 0.798 | |
| ResidualModel=DeiT, Source=feat2022.03 | 0.7454 | 0.9125 | |
| DDUBackbone=ResNet-50, Single-pass uncertainty estimation=true2022.02 | 0.741 | — | |
| DUQBackbone=ResNet-50, Single-pass uncertainty estimation=true2022.02 | 0.714 | — | |
| KL MatchingModel=DeiT, Source=prob2022.03 | 0.7105 | 0.846 | |
| ViMBackbone=ResNet-502026.02 | 0.703 | 0.85 | |
| ReActModel=Res50d, Source=feat2022.03 | 0.6802 | 0.7845 | |
| ReActModel=BiT-S-R101x1, Source=feat+logit2022.03 | 0.6707 | 0.917 | |
| KL MatchingModel=Res50d, Source=prob2022.03 | 0.67 | 0.885 | |
| KL MatchingModel=RepVGG, Source=prob2022.03 | 0.6636 | 0.8495 | |
| KL MatchingModel=BiT-S-R101x1, Source=prob2022.03 | 0.6568 | 0.8665 | |
| ReActModel=DeiT, Source=feat2022.03 | 0.6432 | 0.8185 | |
| MSPModel=DeiT, Source=prob2022.03 | 0.6365 | 0.872 | |
| EnergyModel=BiT-S-R101x1, Source=logit2022.03 | 0.6359 | 0.964 | |
| EnergyModel=RepVGG, Source=logit2022.03 | 0.6336 | 0.8775 | |
| MaxLogitModel=BiT-S-R101x1, Source=logit2022.03 | 0.6301 | 0.9685 | |
| ODINModel=BiT-S-R101x1, Source=prob+grad2022.03 | 0.63 | 0.9685 | |
| MaxLogitModel=RepVGG, Source=logit2022.03 | 0.6287 | 0.899 | |
| ODINModel=RepVGG, Source=prob+grad2022.03 | 0.625 | 0.897 | |
| ODINModel=DeiT, Source=prob+grad2022.03 | 0.617 | 0.8495 | |
| MSPModel=RepVGG, Source=prob2022.03 | 0.6165 | 0.913 | |
| MaxLogitModel=DeiT, Source=logit2022.03 | 0.6138 | 0.837 | |
| EnergyModel=DeiT, Source=logit2022.03 | 0.606 | 0.8275 | |
| EnergyBackbone=ResNet-50, Single-pass uncertainty estimation=true2022.02 | 0.6 | — | |
| FDBDBackbone=ResNet-502026.02 | 0.593 | 1 | |
| MSPModel=BiT-S-R101x1, Source=prob2022.03 | 0.5712 | 0.9685 | |
| MSPModel=Res50d, Source=prob2022.03 | 0.5613 | 0.9385 | |
| MaxLogitModel=Res50d, Source=logit2022.03 | 0.5439 | 0.9265 | |
| EnergyModel=Res50d, Source=logit2022.03 | 0.5395 | 0.901 | |
| ODINModel=Res50d, Source=prob+grad2022.03 | 0.5287 | 0.9325 | |
| EnsembleBackbone=ResNet-50, Single-pass uncertainty estimation=false, Number of models=52022.02 | 0.519 | — | |
| ReActModel=RepVGG, Source=feat2022.03 | 0.4876 | 0.9865 | |
| ReActBackbone=ResNet-502026.02 | 0.465 | 0.999 | |
| EnergyBackbone=ResNet-502026.02 | 0.417 | 1 | |
| MaxLogitBackbone=ResNet-502026.02 | 0.406 | 1 | |
| TTABackbone=ResNet-50, Single-pass uncertainty estimation=false2022.02 | 0.305 | — | |
| MSPBackbone=ResNet-502026.02 | 0.294 | 1 | |
| EntropyBackbone=ResNet-50, Single-pass uncertainty estimation=true2022.02 | 0.291 | — | |
| MaxProbBackbone=ResNet-50, Single-pass uncertainty estimation=true2022.02 | 0.282 | — |