Conformal Prediction on CIFAR-10 (test)
1.09Mean Prediction Set SizeEC3
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| EC3Backbone=DenseNet100, alpha=0.12025.12 | 1.09 | 90 | — | — | — | — | — | — | |
| BPS (1 − α)calibration=none2025.05 | 1.09 | 90 | — | — | — | — | — | 52 | |
| Calibrated BPS with zero-ordercalibration=zero-order data2025.05 | 1.09 | 90 | — | — | — | — | — | 52 | |
| EC3Backbone=PreResNet110, alpha=0.12025.12 | 1.18 | 90 | — | — | — | — | — | — | |
| EC3Backbone=ResNet56, alpha=0.12025.12 | 1.23 | 90 | — | — | — | — | — | — | |
| ConfTrBackbone=PreResNet110, alpha=0.12025.12 | 1.25 | 90 | — | — | — | — | — | — | |
| ConfTrBackbone=DenseNet100, alpha=0.12025.12 | 1.3 | 90 | — | — | — | — | — | — | |
| ConfTrBackbone=ResNet56, alpha=0.12025.12 | 1.31 | 90 | — | — | — | — | — | — | |
| Calibrated BPS with first-ordercalibration=first-order data2025.05 | 1.45 | 98 | — | — | — | — | — | 90 | |
| DPI boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 1.64 | — | — | — | — | — | — | — | |
| Model-based Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 1.66 | — | — | — | — | — | — | — | |
| CEScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 1.69 | — | — | — | — | — | — | — | |
| Model-based Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 1.89 | — | — | — | — | — | — | — | |
| DPI boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 1.97 | — | — | — | — | — | — | — | |
| Simple Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 2.05 | — | — | — | — | — | — | — | |
| ConfTr_classScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 2.16 | — | — | — | — | — | — | — | |
| ConfTr_classScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 2.18 | — | — | — | — | — | — | — | |
| CEScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 2.34 | — | — | — | — | — | — | — | |
| Simple Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 2.35 | — | — | — | — | — | — | — | |
| CPBackbone=ResNet56, alpha=0.12025.12 | 5.41 | 90 | — | — | — | — | — | — | |
| CPBackbone=DenseNet100, alpha=0.12025.12 | 5.52 | 90 | — | — | — | — | — | — | |
| CPBackbone=PreResNet110, alpha=0.12025.12 | 5.54 | 90 | — | — | — | — | — | — | |
| ConfTrScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 9.9 | — | — | — | — | — | — | — | |
| ConfTrScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 9.98 | — | — | — | — | — | — | — | |
| BLModel Type=Aggregation Method2025.12 | — | — | 95.1 | 89.9 | 1.531 | 1.13 | — | — | |
| CMModel Type=Aggregation Method2025.12 | — | — | 98.8 | 97.1 | 2.089 | 1.57 | — | — | |
| CRModel Type=Aggregation Method2025.12 | — | — | 94.8 | 90.8 | 1.506 | 1.194 | — | — | |
| CSAModel Type=Aggregation Method2025.12 | — | — | 95 | 89.8 | 1.294 | 1.038 | — | — | |
| DLAModel Type=Base Model2025.12 | — | — | 95 | 89.9 | 1.62 | 1.218 | — | — | |
| EffNetModel Type=Base Model2025.12 | — | — | 94.9 | 90 | 1.929 | 1.366 | — | — | |
| Hard Pseudo-Calibrationsigma=0.452026.02 | — | 64.7 | — | — | — | — | 0.8 | — | |
| Hard Pseudo-Calibrationsigma=1.052026.02 | — | 43.7 | — | — | — | — | 0.8 | — | |
| Hard Pseudo-Calibrationsigma=1.52026.02 | — | 29.9 | — | — | — | — | 0.8 | — | |
| RN56Model Type=Base Model2025.12 | — | — | 94.9 | 90.1 | 1.833 | 1.352 | — | — | |
| SACPModel Type=Aggregation Method2025.12 | — | — | 95 | 89.9 | 1.308 | 1.034 | — | — | |
| SACP++Model Type=Aggregation Method2025.12 | — | — | 94.9 | 89.8 | 1.281 | 1.028 | — | — | |
| ShuffV2Model Type=Base Model2025.12 | — | — | 94.9 | 89.9 | 2.322 | 1.686 | — | — | |
| Source Calibrationsigma=0.452026.02 | — | 72.7 | — | — | — | — | 0.97 | — | |
| Source Calibrationsigma=1.052026.02 | — | 48.1 | — | — | — | — | 0.95 | — | |
| Source Calibrationsigma=1.52026.02 | — | 32.9 | — | — | — | — | 0.94 | — | |
| Source-Tuned Pseudo-Calibrationsigma=0.452026.02 | — | 93.5 | — | — | — | — | 2.12 | — | |
| Source-Tuned Pseudo-Calibrationsigma=1.052026.02 | — | 87.9 | — | — | — | — | 4.32 | — | |
| Source-Tuned Pseudo-Calibrationsigma=1.52026.02 | — | 77.7 | — | — | — | — | 4.83 | — | |
| Target Calibration (Oracle)sigma=0.452026.02 | — | 80.1 | — | — | — | — | 1.18 | — | |
| Target Calibration (Oracle)sigma=1.052026.02 | — | 79.4 | — | — | — | — | 3.05 | — | |
| Target Calibration (Oracle)sigma=1.52026.02 | — | 80.8 | — | — | — | — | 5.03 | — | |
| VGG16Model Type=Base Model2025.12 | — | — | 95.1 | 89.9 | 1.531 | 1.13 | — | — | |
| WaggModel Type=Aggregation Method2025.12 | — | — | 94.8 | 89.8 | 1.292 | 1.032 | — | — |