Trustworthiness Prediction on ImageNet 1k (val)
93.01AUPRSS
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| SSOracle=ViT, Classifier=ViT2021.09 | 93.01 | 83.9 | 80.48 | 10.26 | 73.68 | 87.52 | 38.27 | |
| SSOracle=RSN, Classifier=ViT2021.09 | 90.01 | 83.9 | 88.7 | 11.69 | 64.34 | 96.2 | 9 | |
| TCPOracle=ViT, Classifier=ViT2021.09 | 87.63 | 83.9 | 88.38 | 12.96 | 60.14 | 99.73 | 0 | |
| TCPOracle=RSN, Classifier=ViT2021.09 | 86.12 | 83.9 | 91.93 | 14.12 | 55.55 | 100 | 0 | |
| CEOracle=RSN, Classifier=ViT2021.09 | 85.4 | 83.9 | 93.29 | 14.74 | 53.43 | 100 | 0 | |
| SSOracle=ViT, Classifier=RSN2021.09 | 85.39 | 68.72 | 77.89 | 20.91 | 74.31 | 68.32 | 67.53 | |
| FocalOracle=RSN, Classifier=ViT2021.09 | 85.13 | 83.9 | 94.6 | 14.98 | 52.37 | 100 | 0 | |
| FocalOracle=ViT, Classifier=ViT2021.09 | 84.76 | 83.9 | 93.37 | 15.31 | 52.38 | 99.15 | 1.35 | |
| CEOracle=ViT, Classifier=ViT2021.09 | 84.25 | 83.9 | 93.01 | 15.8 | 51.62 | 99.99 | 0.02 | |
| SSOracle=RSN, Classifier=RSN2021.09 | 81.88 | 68.72 | 86.21 | 22.53 | 67.92 | 79.2 | 42.09 | |
| TCPOracle=ViT, Classifier=RSN2021.09 | 79.04 | 68.72 | 83.55 | 23.56 | 66.23 | 94.25 | 0 | |
| TCPOracle=RSN, Classifier=RSN2021.09 | 77.79 | 68.72 | 88.81 | 24.46 | 62.73 | 99.23 | 0 | |
| CEOracle=RSN, Classifier=RSN2021.09 | 70.79 | 68.72 | 94.84 | 29.41 | 52.36 | 100 | 0 | |
| FocalOracle=RSN, Classifier=RSN2021.09 | 70.23 | 68.72 | 95.16 | 29.92 | 51.43 | 99.86 | 0.08 | |
| CEOracle=ViT, Classifier=RSN2021.09 | 69.13 | 68.72 | 93.43 | 30.9 | 51.24 | 99.9 | 0.2 | |
| FocalOracle=ViT, Classifier=RSN2021.09 | 69.07 | 68.72 | 93.94 | 30.97 | 51.26 | 93.66 | 7.71 | |
| LogitDynamicsNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 64.58 | — | — | — | — | — | — | |
| Top-K logitsNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 60.98 | — | — | — | — | — | — | |
| EntropyNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 57.66 | — | — | — | — | — | — | |
| MarginNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 54.71 | — | — | — | — | — | — | |
| Linear probingNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 54.2 | — | — | — | — | — | — | |
| Max logitNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 53.95 | — | — | — | — | — | — | |
| ACT-ViTNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 53.9 | — | — | — | — | — | — | |
| Energy scoreNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 41.68 | — | — | — | — | — | — | |
| MahalanobisNumber of classes=1000, Dataset size=50,000, Misclassification rate=20.34%2026.04 | 32.44 | — | — | — | — | — | — |