Probability Calibration on CIFAR-10H K=10 (test)
0.7ECEOracle TS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Oracle TSBackbone=ViT-B/16, Temperature (T)=3.092026.03 | 0.7 | 0.63 | 0.42 | 0.101 | 0.277 | |
| IR-SoftBackbone=ResNet-502026.03 | 0.72 | 0.83 | 0.45 | 0.115 | 0.34 | |
| Dirichlet-SoftBackbone=ViT-B/162026.03 | 0.72 | 0.41 | 0.25 | 0.1 | 0.255 | |
| MCTS S=1Backbone=ViT-B/16, Temperature (T)=3.072026.03 | 0.81 | 0.65 | 0.42 | 0.101 | 0.277 | |
| SLTSBackbone=ViT-B/16, Temperature (T)=3.072026.03 | 0.85 | 0.56 | 0.39 | 0.102 | 0.278 | |
| SoftPlattBackbone=ViT-B/162026.03 | 0.88 | 0.47 | 0.24 | 0.101 | 0.272 | |
| IR-SoftBackbone=ViT-B/162026.03 | 0.91 | 0.61 | 0.43 | 0.105 | 0.321 | |
| VSBackbone=ViT-B/162026.03 | 0.92 | 0.5 | 0.32 | 0.101 | 0.274 | |
| Dirichlet-SoftBackbone=ResNet-502026.03 | 1.25 | 1.06 | 0.35 | 0.109 | 0.271 | |
| VSBackbone=ResNet-502026.03 | 1.35 | 1.26 | 0.39 | 0.109 | 0.289 | |
| MCTS S=1Backbone=ResNet-50, Temperature (T)=3.142026.03 | 1.45 | 1.44 | 0.45 | 0.111 | 0.296 | |
| Oracle TSBackbone=ResNet-50, Temperature (T)=3.172026.03 | 1.5 | 1.52 | 0.45 | 0.111 | 0.296 | |
| SLTSBackbone=ResNet-50, Temperature (T)=3.182026.03 | 1.51 | 1.39 | 0.45 | 0.11 | 0.293 | |
| SoftPlattBackbone=ResNet-502026.03 | 1.52 | 1.31 | 0.35 | 0.11 | 0.288 | |
| LS-TSBackbone=ResNet-50, Temperature (T)=3.082026.03 | 1.57 | 1.31 | 0.46 | 0.109 | 0.293 | |
| LS-TSBackbone=ViT-B/16, Temperature (T)=2.762026.03 | 2.37 | 2.21 | 0.53 | 0.103 | 0.285 | |
| TSBackbone=ResNet-50, Temperature (T)=2.032026.03 | 4.29 | 4.25 | 0.91 | 0.112 | 0.363 | |
| PlattBackbone=ResNet-502026.03 | 4.29 | 4.23 | 0.91 | 0.112 | 0.372 | |
| ATSBackbone=ResNet-502026.03 | 4.4 | 4.37 | 0.94 | 0.113 | 0.35 | |
| Dirichlet-HardBackbone=ResNet-502026.03 | 4.46 | 4.44 | 0.95 | 0.114 | 0.395 | |
| TSBackbone=ViT-B/16, Temperature (T)=2.042026.03 | 4.48 | 4.4 | 0.93 | 0.106 | 0.346 | |
| ATSBackbone=ViT-B/162026.03 | 4.54 | 4.47 | 0.94 | 0.106 | 0.345 | |
| PlattBackbone=ViT-B/162026.03 | 4.54 | 4.38 | 0.93 | 0.105 | 0.363 | |
| Dirichlet-HardBackbone=ViT-B/162026.03 | 4.7 | 4.62 | 0.97 | 0.107 | 0.394 | |
| UncalibratedBackbone=ResNet-502026.03 | 4.97 | 5.71 | 1.09 | 0.12 | 0.692 | |
| UncalibratedBackbone=ViT-B/162026.03 | 4.99 | 5.36 | 1.06 | 0.111 | 0.678 |