Model Calibration on CIFAR-100
1.1ECEPuzzleMix
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PuzzleMixBackbone=ResNet18, Training=from scratch2026.06 | 1.1 | — | — | — | — | — | — | — | 8.82 | 301 | — | — | — | — | — | |
| GCEModel=Wide-ResNet2026.04 | 1.25 | — | — | — | — | — | — | 6.83 | — | — | — | — | — | — | — | |
| DFLModel=Wide-ResNet2026.04 | 1.26 | — | — | — | — | — | — | 3.45 | — | — | — | — | — | — | — | |
| AFLModel=Wide-ResNet2026.04 | 1.26 | — | — | — | — | — | — | 2.29 | — | — | — | — | — | — | — | |
| AFLModel=ResNet-1102026.04 | 1.29 | — | — | — | — | — | — | 1.46 | — | — | — | — | — | — | — | |
| Brier LossModel=Wide-ResNet2026.04 | 1.32 | — | — | — | — | — | — | 3.97 | — | — | — | — | — | — | — | |
| FLSDModel=Wide-ResNet2026.04 | 1.33 | — | — | — | — | — | — | 2.47 | — | — | — | — | — | — | — | |
| AFLModel=ResNet-502026.04 | 1.39 | — | — | — | — | — | — | 1.71 | — | — | — | — | — | — | — | |
| GCEModel=DenseNet-1212026.04 | 1.48 | — | — | — | — | — | — | 10.75 | — | — | — | — | — | — | — | |
| DFLModel=ResNet-1102026.04 | 1.49 | — | — | — | — | — | — | 4.28 | — | — | — | — | — | — | — | |
| GCEModel=ResNet-502026.04 | 1.5 | — | — | — | — | — | — | 11.23 | — | — | — | — | — | — | — | |
| AFLModel=DenseNet-1212026.04 | 1.51 | — | — | — | — | — | — | 1.71 | — | — | — | — | — | — | — | |
| Brier LossModel=ResNet-502026.04 | 1.52 | — | — | — | — | — | — | 4.73 | — | — | — | — | — | — | — | |
| MMCEModel=ResNet-502026.04 | 1.53 | — | — | — | — | — | — | 15.52 | — | — | — | — | — | — | — | |
| DFLModel=ResNet-502026.04 | 1.53 | — | — | — | — | — | — | 2.06 | — | — | — | — | — | — | — | |
| GCEModel=ResNet-1102026.04 | 1.54 | — | — | — | — | — | — | 12.69 | — | — | — | — | — | — | — | |
| FLSDModel=DenseNet-1212026.04 | 1.55 | — | — | — | — | — | — | 3.31 | — | — | — | — | — | — | — | |
| CEModel=Wide-ResNet2026.04 | 1.56 | — | — | — | — | — | — | 15.12 | — | — | — | — | — | — | — | |
| DFLModel=DenseNet-1212026.04 | 1.56 | — | — | — | — | — | — | 4.79 | — | — | — | — | — | — | — | |
| Brier LossModel=DenseNet-1212026.04 | 1.58 | — | — | — | — | — | — | 4.21 | — | — | — | — | — | — | — | |
| FLSDModel=ResNet-1102026.04 | 1.6 | — | — | — | — | — | — | 6.94 | — | — | — | — | — | — | — | |
| Brier LossModel=ResNet-1102026.04 | 1.65 | — | — | — | — | — | — | 6.04 | — | — | — | — | — | — | — | |
| FLSDModel=ResNet-502026.04 | 1.69 | — | — | — | — | — | — | 5.4 | — | — | — | — | — | — | — | |
| MMCEModel=Wide-ResNet2026.04 | 1.73 | — | — | — | — | — | — | 14 | — | — | — | — | — | — | — | |
| OA-CutMix/abs + decoupledBackbone=ResNet18, Training=from scratch2026.06 | 1.74 | — | — | — | — | — | — | — | 4.85 | 1.01 | — | — | — | — | — | |
| CEModel=ResNet-502026.04 | 1.75 | — | — | — | — | — | — | 17.49 | — | — | — | — | — | — | — | |
| OA-CutMix/absBackbone=ResNet18, Training=from scratch2026.06 | 1.79 | — | — | — | — | — | — | — | 4.27 | 1.05 | — | — | — | — | — | |
| ResizeMixBackbone=ResNet18, Training=from scratch2026.06 | 1.93 | — | — | — | — | — | — | — | 5.04 | 0.62 | — | — | — | — | — | |
| MMCEModel=DenseNet-1212026.04 | 1.94 | — | — | — | — | — | — | 17.08 | — | — | — | — | — | — | — | |
| CEModel=ResNet-1102026.04 | 2.06 | — | — | — | — | — | — | 18.78 | — | — | — | — | — | — | — | |
| CEModel=DenseNet-1212026.04 | 2.09 | — | — | — | — | — | — | 19.27 | — | — | — | — | — | — | — | |
| MMCEModel=ResNet-1102026.04 | 2.11 | — | — | — | — | — | — | 18.62 | — | — | — | — | — | — | — | |
| CutMix + decoupledBackbone=DeiT-S, latency [ms]=0.672026.06 | 2.43 | — | — | — | — | — | — | — | 4.71 | — | — | — | — | — | — | |
| OA-CutMix/relBackbone=ResNet18, Training=from scratch2026.06 | 2.56 | — | — | — | — | — | — | — | 4.76 | 1.02 | — | — | — | — | — | |
| OA-CutMix/rel + decoupledBackbone=ResNet18, Training=from scratch2026.06 | 2.77 | — | — | — | — | — | — | — | 4.66 | 1 | — | — | — | — | — | |
| CutMix + decoupledBackbone=ResNet18, Training=from scratch2026.06 | 2.88 | — | — | — | — | — | — | — | 7.42 | 0.6 | — | — | — | — | — | |
| OA-CutMix/abs + decoupledBackbone=DeiT-Ti, latency [ms]=1.012026.06 | 3.01 | — | — | — | — | — | — | — | 3.66 | — | — | — | — | — | — | |
| CutMix + decoupledBackbone=DeiT-Ti, latency [ms]=0.672026.06 | 3.04 | — | — | — | — | — | — | — | 4.71 | — | — | — | — | — | — | |
| SnapMixBackbone=ResNet18, Training=from scratch2026.06 | 3.1 | — | — | — | — | — | — | — | 6.17 | 51.49 | — | — | — | — | — | |
| OA-CutMix/absBackbone=DeiT-Ti, latency [ms]=1.012026.06 | 3.18 | — | — | — | — | — | — | — | 3.27 | — | — | — | — | — | — | |
| OA-CutMix/absBackbone=ResNet50, Training=from scratch2026.06 | 3.37 | — | — | — | — | — | — | — | 4.27 | 1.05 | — | — | — | — | — | |
| OA-CutMix/relBackbone=DeiT-S, latency [ms]=1.032026.06 | 3.39 | — | — | — | — | — | — | — | 3.96 | — | — | — | — | — | — | |
| AlignmixupBackbone=ResNet50, Training=from scratch2026.06 | 3.61 | — | — | — | — | — | — | — | 6.13 | 1.98 | — | — | — | — | — | |
| OA-CutMix/relBackbone=ResNet50, Training=from scratch2026.06 | 3.69 | — | — | — | — | — | — | — | 4.76 | 1.02 | — | — | — | — | — | |
| OA-CutMix/relBackbone=DeiT-Ti, latency [ms]=1.032026.06 | 3.77 | — | — | — | — | — | — | — | 3.96 | — | — | — | — | — | — | |
| OA-CutMix/absBackbone=DeiT-S, latency [ms]=1.012026.06 | 3.78 | — | — | — | — | — | — | — | 3.27 | — | — | — | — | — | — | |
| OA-CutMix/rel + decoupledBackbone=DeiT-S, latency [ms]=1.112026.06 | 3.83 | — | — | — | — | — | — | — | 4.6 | — | — | — | — | — | — | |
| FMixBackbone=DeiT-Ti, latency [ms]=6.802026.06 | 3.98 | — | — | — | — | — | — | — | 6.91 | — | — | — | — | — | — | |
| AlignmixupBackbone=ResNet18, Training=from scratch2026.06 | 4 | — | — | — | — | — | — | — | 6.13 | 1.98 | — | — | — | — | — | |
| IsotonicBackbone=ViT-B16, Alpha (α)=0.01, Nonconformity score=MSP, Transformation=constant2025.05 | 4 | — | — | — | 0.21 | — | — | — | — | — | 0 | 1.067 | 16 | 1.4 | 2.4 | |
| Platt ScalingBackbone=ViT-B16, Alpha (α)=0.01, Nonconformity score=MSP, Transformation=constant2025.05 | 4 | — | — | — | 0.49 | — | — | — | — | — | 0 | 0.404 | 16 | 0.7 | 0.8 | |
| Temp. scalingBackbone=ViT-B16, Alpha (α)=0.01, Nonconformity score=MSP, Transformation=constant2025.05 | 4 | — | — | — | 0.3 | — | — | — | — | — | 0 | 0.446 | 16 | 2.7 | 1 | |
| PuzzleMixBackbone=ResNet50, Training=from scratch2026.06 | 4.05 | — | — | — | — | — | — | — | 8.82 | 301 | — | — | — | — | — | |
| FMixBackbone=ResNet50, Training=from scratch2026.06 | 4.12 | — | — | — | — | — | — | — | 7.65 | 6.68 | — | — | — | — | — | |
| FMixBackbone=ResNet18, Training=from scratch2026.06 | 4.15 | — | — | — | — | — | — | — | 7.65 | 6.68 | — | — | — | — | — | |
| BalCALBackbone=WRN-28-102025.04 | 4.21 | — | — | — | — | 73.21 | 4.13 | — | — | — | — | — | — | — | — | |
| OA-CutMix/abs + decoupledBackbone=DeiT-S, latency [ms]=1.012026.06 | 4.35 | — | — | — | — | — | — | — | 3.66 | — | — | — | — | — | — | |
| PuzzleMixBackbone=DeiT-S, latency [ms]=273.282026.06 | 4.38 | — | — | — | — | — | — | — | 6.1 | — | — | — | — | — | — | |
| MixupBackbone=WRN-28-102025.04 | 4.51 | — | — | — | — | 77.05 | 4.58 | — | — | — | — | — | — | — | — | |
| OA-CutMix/rel + decoupledBackbone=ResNet50, Training=from scratch2026.06 | 4.53 | — | — | — | — | — | — | — | 4.66 | 1 | — | — | — | — | — | |
| ResizeMixBackbone=DeiT-S, latency [ms]=1.052026.06 | 4.64 | — | — | — | — | — | — | — | 7.44 | — | — | — | — | — | — | |
| MergeMixBackbone=ViT-Large, Epochs=200, token merging=false2025.10 | 4.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MergeMixBackbone=ViT-Large, Epochs=200, token merging=true2025.10 | 4.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MixUpBackbone=ViT-Large, Epochs=200, token merging=false2025.10 | 4.76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OA-CutMix/rel + decoupledBackbone=DeiT-Ti, latency [ms]=1.112026.06 | 4.76 | — | — | — | — | — | — | — | 4.6 | — | — | — | — | — | — | |
| FMixBackbone=DeiT-S, latency [ms]=6.802026.06 | 4.89 | — | — | — | — | — | — | — | 6.91 | — | — | — | — | — | — | |
| BaseBackbone=ViT-B16, Alpha (α)=0.01, Nonconformity score=MSP, Transformation=constant2025.05 | 5 | — | — | — | 0.34 | — | — | — | — | — | 0 | 0.437 | 17 | 0.3 | 0.1 | |
| ResizeMixBackbone=ResNet50, Training=from scratch2026.06 | 5.17 | — | — | — | — | — | — | — | 5.04 | 0.62 | — | — | — | — | — | |
| TokenMixBackbone=DeiT-S, latency [ms]=39.912026.06 | 5.18 | — | — | — | — | — | — | — | 7.52 | — | — | — | — | — | — | |
| MergeMixBackbone=DeiT-Tiny, Epochs=600, token merging=false2025.10 | 5.45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MergeMixBackbone=DeiT-Tiny, Epochs=600, token merging=true2025.10 | 5.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MergeMixBackbone=ViT-Small, Epochs=600, token merging=false2025.10 | 5.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=ResNet18, Training=from scratch2026.06 | 5.6 | — | — | — | — | — | — | — | 9.33 | — | — | — | — | — | — | |
| Attentive CutMixBackbone=DeiT-S, latency [ms]=56.102026.06 | 5.61 | — | — | — | — | — | — | — | 7.64 | — | — | — | — | — | — | |
| OA-CutMix/abs + decoupledBackbone=ResNet50, Training=from scratch2026.06 | 5.63 | — | — | — | — | — | — | — | 4.85 | 1.01 | — | — | — | — | — | |
| SnapMixBackbone=ResNet50, Training=from scratch2026.06 | 5.73 | — | — | — | — | — | — | — | 6.17 | 51.49 | — | — | — | — | — | |
| TokenMixBackbone=DeiT-Ti, latency [ms]=39.912026.06 | 5.74 | — | — | — | — | — | — | — | 7.52 | — | — | — | — | — | — | |
| Attentive CutMixBackbone=DeiT-Ti, latency [ms]=56.102026.06 | 5.96 | — | — | — | — | — | — | — | 7.64 | — | — | — | — | — | — | |
| Naive CMCEBackbone=ViT-B16, Alpha (α)=0.01, Nonconformity score=MSP, Transformation=constant2025.05 | 6 | — | — | — | 0.32 | — | — | — | — | — | 0 | 0.456 | 17 | 0.1 | 0.9 | |
| TSBackbone=ViT-B16, Alpha (α)=0.01, Nonconformity score=MSP, Transformation=constant2025.05 | 6 | — | — | — | 0.43 | — | — | — | — | — | 0 | 0.455 | 17 | 0.4 | 0 | |
| CutMixBackbone=DeiT-Tiny, Epochs=600, token merging=false2025.10 | 6.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LSBackbone=WRN-28-102025.04 | 6.03 | — | — | — | — | 73.58 | 6.03 | — | — | — | — | — | — | — | — | |
| MixupBackbone=ResNet50, Training=from scratch2026.06 | 6.03 | — | — | — | — | — | — | — | 12.18 | 0.73 | — | — | — | — | — | |
| MergeMixBackbone=ViT-Small, Epochs=600, token merging=true2025.10 | 6.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CutMixBackbone=ResNet18, Training=from scratch2026.06 | 6.14 | — | — | — | — | — | — | — | 9.08 | 0.61 | — | — | — | — | — | |
| MixProBackbone=ViT-Large, Epochs=200, token merging=false2025.10 | 6.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MixupBackbone=DeiT-S, latency [ms]=0.732026.06 | 6.36 | — | — | — | — | — | — | — | 6.03 | — | — | — | — | — | — | |
| MixProBackbone=DeiT-Tiny, Epochs=600, token merging=false2025.10 | 6.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MixUpBackbone=DeiT-Tiny, Epochs=600, token merging=false2025.10 | 6.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CutMixBackbone=ViT-Small, Epochs=600, token merging=false2025.10 | 6.45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixup+CutMixBackbone=ResNet50, Training=from scratch2026.06 | 6.46 | — | — | — | — | — | — | — | 12.25 | 0.67 | — | — | — | — | — | |
| MixupBackbone=DeiT-Ti, latency [ms]=0.732026.06 | 6.49 | — | — | — | — | — | — | — | 6.03 | — | — | — | — | — | — | |
| DeiTBackbone=ViT-Large, Epochs=200, token merging=false2025.10 | 6.71 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MergeMixBackbone=DeiT-Tiny, Epochs=200, token merging=true2025.10 | 6.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ada-TSBackbone=ViT-B16, Alpha (α)=0.01, Nonconformity score=MSP, Transformation=constant2025.05 | 7 | — | — | — | 0.36 | — | — | — | — | — | 0 | 0.889 | 18 | 0.8 | 3.5 | |
| TSTBackbone=WRN-28-102025.04 | 7.07 | — | — | — | — | 71.26 | — | — | — | — | — | — | — | — | — | |
| MergeMixBackbone=DeiT-Tiny, Epochs=200, token merging=false2025.10 | 7.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CutMix + decoupledBackbone=ResNet50, Training=from scratch2026.06 | 7.28 | — | — | — | — | — | — | — | 7.42 | 0.6 | — | — | — | — | — | |
| Mixup + CutMixBackbone=DeiT-S, latency [ms]=0.702026.06 | 7.33 | — | — | — | — | — | — | — | 7.41 | — | — | — | — | — | — | |
| VTSTBackbone=WRN-28-102025.04 | 7.34 | — | — | — | — | 69.18 | — | — | — | — | — | — | — | — | — |