Selective Classification on CIFAR-100 (test)
0.9381AUCCE+AUCOCL
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| CE+AUCOCLLoss function=CE+AUCOCLoss, Backbone=ResNet-502023.08 | 0.9381 | 42.61 | — | 75.94 | 28.39 | — | |
| FL+AUCOCLLoss function=FL+AUCOCLoss, Backbone=ResNet-502023.08 | 0.9319 | 43.11 | — | 75.9 | 29.45 | — | |
| FL+ S-AvUCLoss function=FL+ S-AvUC, Backbone=ResNet-502023.08 | 0.9271 | 44.51 | — | 74.82 | 31.11 | — | |
| FL (γ=3)Loss function=Focal Loss, Backbone=ResNet-502023.08 | 0.9268 | 46.4 | — | 75.13 | 32.3 | — | |
| AdaFL53Loss function=AdaFL53, Backbone=ResNet-502023.08 | 0.9265 | 45.78 | — | 74.5 | 32.01 | — | |
| CE+S-AvUCLoss function=CE+S-AvUC, Backbone=ResNet-502023.08 | 0.9263 | 44.83 | — | 74.12 | 31.02 | — | |
| FL+S-ECELoss function=FL+S-ECE, Backbone=ResNet-502023.08 | 0.9263 | 47.42 | — | 75 | 31.35 | — | |
| CE+S-ECELoss function=CE+S-ECE, Backbone=ResNet-502023.08 | 0.9257 | 45.4 | — | 74.27 | 31.38 | — | |
| CE+MMCELoss function=CE+MMCE, Backbone=ResNet-502023.08 | 0.9255 | 44.53 | — | 74.75 | 31.08 | — | |
| FL+MMCELoss function=FL+MMCE, Backbone=ResNet-502023.08 | 0.9246 | 46.63 | — | 74.47 | 31.69 | — | |
| CELoss function=Cross-Entropy, Backbone=ResNet-502023.08 | 0.911 | 49.42 | — | 73.69 | 34.72 | — | |
| Fisher-EDLUncertainty measure=Max Probability (MaxP)2024.02 | 0.861 | — | — | 55.9 | — | 80.1 | |
| Fisher-EDLUncertainty measure=Entropy (Ent)2024.02 | 0.858 | — | — | 55.9 | — | 79.6 | |
| Bootstrap DistillUncertainty measure=Max Probability (MaxP)2024.02 | 0.857 | — | — | 65.4 | — | 73.2 | |
| END2Uncertainty measure=Max Probability (MaxP)2024.02 | 0.853 | — | — | 65 | — | 72.6 | |
| EDLUncertainty measure=Max Probability (MaxP)2024.02 | 0.849 | — | — | 62.9 | — | 74.9 | |
| EDLUncertainty measure=Entropy (Ent)2024.02 | 0.845 | — | — | 62.9 | — | 74.1 | |
| S2DUncertainty measure=Max Probability (MaxP)2024.02 | 0.844 | — | — | 61 | — | 74.2 | |
| S2DUncertainty measure=Entropy (Ent)2024.02 | 0.834 | — | — | 61 | — | 72.7 | |
| Bootstrap DistillUncertainty measure=Entropy (Ent)2024.02 | 0.832 | — | — | 65.4 | — | 69.4 | |
| END2Uncertainty measure=Entropy (Ent)2024.02 | 0.83 | — | — | 65 | — | 69.3 | |
| BMUncertainty measure=Max Probability (MaxP)2024.02 | 0.826 | — | — | 54.3 | — | 76.9 | |
| PostNetUncertainty measure=Max Probability (MaxP)2024.02 | 0.826 | — | — | 58 | — | 75.4 | |
| NatPNUncertainty measure=Max Probability (MaxP)2024.02 | 0.822 | — | — | 56 | — | 76.2 | |
| RPriorNetUncertainty measure=Max Probability (MaxP)2024.02 | 0.818 | — | — | 57.9 | — | 74 | |
| BMUncertainty measure=Entropy (Ent)2024.02 | 0.818 | — | — | 54.3 | — | 76 | |
| PostNetUncertainty measure=Entropy (Ent)2024.02 | 0.816 | — | — | 58 | — | 74.1 | |
| RPriorNetUncertainty measure=Entropy (Ent)2024.02 | 0.808 | — | — | 57.9 | — | 72.2 | |
| NatPNUncertainty measure=Entropy (Ent)2024.02 | 0.808 | — | — | 56 | — | 73.8 | |
| RRattack=PGD-1000, restarts=10, step size=2/255, perturbation constraint=8/255, threat model=l_inf2021.05 | 0.776 | 34.48 | — | — | — | — | |
| EBDattack=PGD-1000, restarts=10, step size=2/255, perturbation constraint=8/255, threat model=l_inf2021.05 | 0.769 | 33.35 | — | — | — | — | |
| SNetattack=PGD-1000, restarts=10, step size=2/255, perturbation constraint=8/255, threat model=l_inf2021.05 | 0.744 | 32.69 | — | — | — | — | |
| Deep GamblerCoverage=100%2026.02 | — | — | 21.7 | — | — | — | |
| SelectiveNetCoverage (%)=100, Imbalance Factor=2002022.02 | — | — | 68.74 | — | — | — | |
| SelectiveNetCoverage (%)=90, Imbalance Factor=202022.02 | — | — | 51.12 | — | — | — | |
| SelectiveNetCoverage (%)=70, Imbalance Factor=02022.02 | — | — | 14.63 | — | — | — | |
| SelectiveNetCoverage=100%2026.02 | — | — | 21.68 | — | — | — | |
| SelectiveNet + CMW-NetCoverage (%)=100, Imbalance Factor=2002022.02 | — | — | 65.62 | — | — | — | |
| SelectiveNet + CMW-NetCoverage (%)=90, Imbalance Factor=202022.02 | — | — | 42.24 | — | — | — | |
| SelectiveNet + CMW-NetCoverage (%)=70, Imbalance Factor=02022.02 | — | — | 10.69 | — | — | — | |
| SYNCγ=1Coverage=100%2026.02 | — | — | 21.22 | — | — | — | |
| VanillaCoverage=100%2026.02 | — | — | 22.27 | — | — | — |