Model Calibration on Tiny-ImageNet (test)
1.22ECEFeDLaS-MbLS
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FeDLaS-MbLSArch=ResNet502026.06 | 1.22 | — | — | — | — | 1.11 | |
| AdaFL53Loss function=AdaFL53, Post temperature scaling=true2023.08 | 1.35 | 0.85 | 65.71 | 0.159 | — | — | |
| ACLSArch=ResNet502026.06 | 1.37 | — | — | — | — | 1.36 | |
| MbLSArch=ResNet502026.06 | 1.45 | — | — | — | — | 1.47 | |
| FL (γ=3)Loss function=FL (γ=3), Post temperature scaling=true2023.08 | 1.46 | 1.14 | 65.91 | 0.152 | — | — | |
| BLBackbone=ResNet50, Temperature Scaling=Post T2025.03 | 1.47 | — | — | — | — | — | |
| BSCE-GRABackbone=ResNet50, Temperature Scaling=Post T2025.03 | 1.47 | — | — | — | — | — | |
| FL+S-AvUCLoss function=FL+S-AvUC, Post temperature scaling=true2023.08 | 1.51 | 1.13 | 64.62 | 0.15 | — | — | |
| CELoss function=CE, Post temperature scaling=true2023.08 | 1.54 | 0.74 | 66.25 | 0.163 | — | — | |
| FeDLaS-MbLSArch=ResNet1012026.06 | 1.55 | — | — | — | — | 1.39 | |
| CE+ TSArch=ResNet502026.06 | 1.63 | — | — | — | — | 1.52 | |
| CE+AUCOCLLoss function=CE+AUCOCL, Post temperature scaling=true2023.08 | 1.65 | 1.29 | 64.23 | 0.157 | — | — | |
| CRLArch=ResNet502026.06 | 1.65 | — | — | — | — | 1.52 | |
| CE+S-ECELoss function=CE+S-ECE, Post temperature scaling=true2023.08 | 1.66 | 0.81 | 66.14 | 0.15 | — | — | |
| ACLSArch=ResNet1012026.06 | 1.66 | — | — | — | — | 1.65 | |
| BSCEBackbone=ResNet50, Temperature Scaling=Post T2025.03 | 1.76 | — | — | — | — | — | |
| FeDLaS-LSArch=ResNet502026.06 | 1.77 | — | — | — | — | 1.65 | |
| CE+MMCELoss function=CE+MMCE, Post temperature scaling=true2023.08 | 1.79 | 0.93 | 66.42 | 0.157 | — | — | |
| MbLSArch=ResNet1012026.06 | 1.81 | — | — | — | — | 1.58 | |
| FL+AUCOCLLoss function=FL+AUCOCL, Post temperature scaling=true2023.08 | 1.86 | 1.29 | 64.45 | 0.155 | — | — | |
| FL+MMCELoss function=FL+MMCE, Post temperature scaling=true2023.08 | 1.94 | 1.63 | 65.49 | 0.151 | — | — | |
| LSArch=ResNet1012026.06 | 2.04 | — | — | — | — | 1.96 | |
| CE+ TSArch=ResNet1012026.06 | 2.08 | — | — | — | — | 2.03 | |
| CE+S-AvUCLoss function=CE+S-AvUC, Post temperature scaling=true2023.08 | 2.13 | 1.15 | 65.2 | 0.146 | — | — | |
| FLSDBackbone=ResNet50, Temperature Scaling=Pre T2025.03 | 2.18 | — | — | — | — | — | |
| FLSDBackbone=ResNet50, Temperature Scaling=Post T2025.03 | 2.18 | — | — | — | — | — | |
| DFLBackbone=ResNet50, Temperature Scaling=Post T2025.03 | 2.28 | — | — | — | — | — | |
| FLArch=ResNet1012026.06 | 2.55 | — | — | — | — | 2.44 | |
| LSArch=ResNet502026.06 | 2.56 | — | — | — | — | 2.58 | |
| FeDLaS-LSArch=ResNet1012026.06 | 2.56 | — | — | — | — | 2.39 | |
| FL+S-ECELoss function=FL+S-ECE, Post temperature scaling=true2023.08 | 2.6 | 2.37 | 66.4 | 0.161 | — | — | |
| MDCAArch=ResNet502026.06 | 2.77 | — | — | — | — | 2.61 | |
| FLSD-53Arch=ResNet502026.06 | 2.91 | — | — | — | — | 2.95 | |
| FLArch=ResNet502026.06 | 2.96 | — | — | — | — | 3.12 | |
| MMCEBackbone=ResNet50, Temperature Scaling=Post T2025.03 | 2.99 | — | — | — | — | — | |
| CPCArch=ResNet502026.06 | 3.12 | — | — | — | — | 3.05 | |
| CRLArch=ResNet1012026.06 | 3.57 | — | — | — | — | 3.56 | |
| CEArch=ResNet502026.06 | 3.73 | — | — | — | — | 3.69 | |
| CPCArch=ResNet1012026.06 | 3.9 | — | — | — | — | 4 | |
| BSCEBackbone=ResNet50, Temperature Scaling=Pre T2025.03 | 4 | — | — | — | — | — | |
| ECPArch=ResNet502026.06 | 4 | — | — | — | — | 3.92 | |
| BSCE-GRABackbone=ResNet50, Temperature Scaling=Pre T2025.03 | 4.57 | — | — | — | — | — | |
| ECPArch=ResNet1012026.06 | 4.68 | — | — | — | — | 4.66 | |
| DFLArch=ResNet1012026.06 | 4.78 | — | — | — | — | 4.77 | |
| MMCEArch=ResNet1012026.06 | 4.88 | — | — | — | — | 4.88 | |
| FLSD-53Arch=ResNet1012026.06 | 4.91 | — | — | — | — | 4.91 | |
| CEArch=ResNet1012026.06 | 4.97 | — | — | — | — | 4.97 | |
| MMCEArch=ResNet502026.06 | 5.15 | — | — | — | — | 5.12 | |
| CEBackbone=ResNet50, Temperature Scaling=Post T2025.03 | 5.16 | — | — | — | — | — | |
| AdaFocalArch=ResNet1012026.06 | 5.43 | — | — | — | — | 5.42 | |
| AdaFocalArch=ResNet502026.06 | 5.82 | — | — | — | — | 5.8 | |
| MDCAArch=ResNet1012026.06 | 6.06 | — | — | — | — | 6.06 | |
| DFLArch=ResNet502026.06 | 6.5 | — | — | — | — | 6.5 | |
| DFLBackbone=ResNet50, Temperature Scaling=Pre T2025.03 | 6.71 | — | — | — | — | — | |
| BLBackbone=ResNet50, Temperature Scaling=Pre T2025.03 | 7.81 | — | — | — | — | — | |
| MMCEBackbone=ResNet50, Temperature Scaling=Pre T2025.03 | 14.58 | — | — | — | — | — | |
| CEBackbone=ResNet50, Temperature Scaling=Pre T2025.03 | 14.94 | — | — | — | — | — | |
| BLModel=ResNet50, Temperature Scaling=Pre T2025.03 | — | — | — | — | 6.8 | — | |
| BLModel=ResNet50, Temperature Scaling=Post T2025.03 | — | — | — | — | 1.38 | — | |
| BSCEModel=ResNet50, Temperature Scaling=Pre T2025.03 | — | — | — | — | 4 | — | |
| BSCEModel=ResNet50, Temperature Scaling=Post T2025.03 | — | — | — | — | 1.7 | — | |
| BSCE-GRAModel=ResNet50, Temperature Scaling=Pre T2025.03 | — | — | — | — | 4.56 | — | |
| BSCE-GRAModel=ResNet50, Temperature Scaling=Post T2025.03 | — | — | — | — | 1.34 | — | |
| CEModel=ResNet50, Temperature Scaling=Pre T2025.03 | — | — | — | — | 14.93 | — | |
| CEModel=ResNet50, Temperature Scaling=Post T2025.03 | — | — | — | — | 5.15 | — | |
| DFLModel=ResNet50, Temperature Scaling=Pre T2025.03 | — | — | — | — | 6.71 | — | |
| DFLModel=ResNet50, Temperature Scaling=Post T2025.03 | — | — | — | — | 2.2 | — | |
| FLSDModel=ResNet50, Temperature Scaling=Pre T2025.03 | — | — | — | — | 1.9 | — | |
| FLSDModel=ResNet50, Temperature Scaling=Post T2025.03 | — | — | — | — | 1.9 | — | |
| MMCEModel=ResNet50, Temperature Scaling=Pre T2025.03 | — | — | — | — | 13.5 | — | |
| MMCEModel=ResNet50, Temperature Scaling=Post T2025.03 | — | — | — | — | 4.92 | — |