Out-of-Distribution Detection on FashionMNIST (ID) vs MNIST (OoD)
0.998AUROCS score
Evaluation results
| Method | Links | |
|---|---|---|
| S scoregenerative_model=Glow, compression_algorithm=FLIF2019.09 | 0.998 | |
| FSVIArchitecture=four-layer CNN, Ensemble=false, Context distribution (p_xc)=KMNIST2023.12 | 0.998 | |
| Likelihood-ratioapproach_type=Generative-based2019.09 | 0.997 | |
| LL ratioCharacteristics=Need to train extra generative models2024.04 | 0.994 | |
| Mahalanobisapproach_type=Classifier-based2019.09 | 0.986 | |
| FSVI EnsembleArchitecture=four-layer CNN, Ensemble=true, Context distribution (p_xc)=random monochrome2023.12 | 0.9785 | |
| LDCharacteristics=No impact on original model2024.04 | 0.971 | |
| S scoregenerative_model=PixelCNN++, compression_algorithm=FLIF2019.09 | 0.967 | |
| FSVIArchitecture=four-layer CNN, Ensemble=false, Context distribution (p_xc)=random monochrome2023.12 | 0.9623 | |
| Likelihood RatioDensity Estimator=Glow2026.02 | 0.9616 | |
| SPGArchitecture=four-layer CNN, Ensemble=false2023.12 | 0.956 | |
| BNN-LAPLACEArchitecture=four-layer CNN, Ensemble=false2023.12 | 0.9555 | |
| DUQCharacteristics=Use particular type of model difficult to train2024.04 | 0.955 | |
| DUQArchitecture=four-layer CNN, Ensemble=false2023.12 | 0.955 | |
| LIDDensity Estimator=Glow2026.02 | 0.951 | |
| Euclidean DistanceCharacteristics=No impact on original model2024.04 | 0.943 | |
| SPEMDensity Estimator=Glow2026.02 | 0.9424 | |
| SPEM-noiseDensity Estimator=Glow2026.02 | 0.9424 | |
| Mahalanobis DistanceCharacteristics=No impact on original model2024.04 | 0.942 | |
| VIBapproach_type=Classifier-based2019.09 | 0.941 | |
| Sketched Lanczos UncertaintyModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.94 | |
| LIN-RIEM-LAPrior Precision=Not Optimized, Exponential Maps Source=Subset of Training Set2023.06 | 0.94 | |
| LIN-LAPrior Precision=Not Optimized2023.06 | 0.937 | |
| RIEM-LAPrior Precision=Optimized, Exponential Maps Source=Subset of Training Set2023.06 | 0.934 | |
| MFVIArchitecture=four-layer CNN, Ensemble=false2023.12 | 0.931 | |
| LIN-RIEM-LAPrior Precision=Not Optimized, Exponential Maps Source=Full Training Set2023.06 | 0.929 | |
| RIEM-LAPrior Precision=Optimized, Exponential Maps Source=Full Training Set2023.06 | 0.921 | |
| Deep EnsembleModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.92 | |
| RIEM-LAPrior Precision=Not Optimized, Exponential Maps Source=Subset of Training Set2023.06 | 0.917 | |
| RIEM-LAPrior Precision=Not Optimized, Exponential Maps Source=Full Training Set2023.06 | 0.895 | |
| Deep EnsembleArchitecture=four-layer CNN, Ensemble=true2023.12 | 0.8922 | |
| MC DROPOUTArchitecture=four-layer CNN, Ensemble=false2023.12 | 0.8846 | |
| MAPArchitecture=four-layer CNN, Ensemble=false2023.12 | 0.87 | |
| LIN-RIEM-LAPrior Precision=Optimized, Exponential Maps Source=Subset of Training Set2023.06 | 0.869 | |
| MC-DropoutUncertainty Metric=Predictive Entropy2021.06 | 0.864 | |
| LIN-LAPrior Precision=Optimized2023.06 | 0.864 | |
| MFVI (tempered)Architecture=four-layer CNN, Ensemble=false2023.12 | 0.863 | |
| Deep EnsemblesCharacteristics=Need to train many models, Number of models=52024.04 | 0.861 | |
| Vanilla LAPrior Precision=Optimized2023.06 | 0.861 | |
| ComplexityDensity Estimator=Glow2026.02 | 0.8604 | |
| SWAGArchitecture=four-layer CNN, Ensemble=false2023.12 | 0.8518 | |
| MFVI (radial)Architecture=four-layer CNN, Ensemble=false2023.12 | 0.844 | |
| SCODModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.84 | |
| LIN-RIEM-LAPrior Precision=Optimized, Exponential Maps Source=Full Training Set2023.06 | 0.807 | |
| LLAModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.8 | |
| LEModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.8 | |
| LE-HModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.8 | |
| GMMDensity Estimator=Glow2026.02 | 0.7964 | |
| ReDecNNUncertainty Metric=Predictive Entropy2021.06 | 0.793 | |
| WAICapproach_type=Generative-based2019.09 | 0.766 | |
| SWAGModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.75 | |
| Typicality (sqrt(d))Density Estimator=Glow2026.02 | 0.7365 | |
| LikelihoodDensity Estimator=Glow2026.02 | 0.7359 | |
| MAPPrior Precision=Not Optimized2023.06 | 0.715 | |
| MAPPrior Precision=Optimized2023.06 | 0.715 | |
| ODINapproach_type=Classifier-based2019.09 | 0.697 | |
| LLA-DModel Architecture=LeNet, Parameters (p)=40K, Memory Budget=3p2024.09 | 0.68 | |
| Typicality (Entropy)Density Estimator=Glow2026.02 | 0.5878 | |
| EnsembleUncertainty Metric=Predictive Entropy, Ensemble Size=D2021.06 | 0.501 | |
| Vanilla LAPrior Precision=Not Optimized2023.06 | 0.494 | |
| Typicality testapproach_type=Generative-based, batch_size=22019.09 | 0.14 |