Conformal Prediction on ImageNet
1.52Average Prediction Set Sizeoracle
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| oracleScore=THR, Number of labeled data=802025.05 | 1.52 | 0.29 | — | — | — | — | — | — | — | — | — | |
| semicpScore=THR, Number of labeled data=802025.05 | 1.62 | 2.63 | — | — | — | — | — | — | — | — | — | |
| standardScore=THR, Number of labeled data=802025.05 | 1.7 | 2.69 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=softmax, n_avg=102023.06 | 1.9 | 5.2 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=softmax, n_avg=202023.06 | 1.9 | 5.2 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=softmax, n_avg=502023.06 | 1.9 | 5.3 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=softmax, n_avg=752023.06 | 1.9 | 5.2 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=softmax, n_avg=102023.06 | 2.5 | 4.9 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=softmax, n_avg=502023.06 | 2.6 | 3.1 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=softmax, n_avg=202023.06 | 2.7 | 3.9 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=softmax, n_avg=752023.06 | 2.7 | 2.8 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=softmax, n_avg=752023.06 | 4.2 | 2.8 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=softmax, n_avg=502023.06 | 5.1 | 3.4 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=RAPS, n_avg=202023.06 | 5.1 | 3 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=RAPS, n_avg=502023.06 | 5.1 | 3 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=RAPS, n_avg=752023.06 | 5.1 | 3 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=RAPS, n_avg=102023.06 | 5.2 | 3 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=RAPS, n_avg=502023.06 | 6.5 | 2.6 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=RAPS, n_avg=202023.06 | 6.6 | 2.9 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=RAPS, n_avg=752023.06 | 6.8 | 2.4 | — | — | — | — | — | — | — | — | — | |
| APS+TTA-AvgAlpha=0.1, Augmentation Policy=Expanded2025.05 | 7.193 | — | — | — | — | — | — | — | — | — | — | |
| APS+TTA-LearnedAlpha=0.1, Augmentation Policy=Expanded2025.05 | 7.215 | — | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=RAPS, n_avg=752023.06 | 7.3 | 2.8 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=RAPS, n_avg=102023.06 | 7.7 | 3.1 | — | — | — | — | — | — | — | — | — | |
| APS+TTA-AvgAlpha=0.1, Augmentation Policy=Simple2025.05 | 8.787 | — | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=RAPS, n_avg=502023.06 | 8.8 | 3.4 | — | — | — | — | — | — | — | — | — | |
| APS+TTA-LearnedAlpha=0.1, Augmentation Policy=Simple2025.05 | 8.813 | — | — | — | — | — | — | — | — | — | — | |
| APSAlpha=0.1, Augmentation Policy=Expanded2025.05 | 8.969 | — | — | — | — | — | — | — | — | — | — | |
| APSAlpha=0.1, Augmentation Policy=Simple2025.05 | 8.969 | — | — | — | — | — | — | — | — | — | — | |
| APS+TTA-LearnedAlpha=0.05, Augmentation Policy=Expanded2025.05 | 14.084 | — | — | — | — | — | — | — | — | — | — | |
| APS+TTA-AvgAlpha=0.05, Augmentation Policy=Expanded2025.05 | 14.308 | — | — | — | — | — | — | — | — | — | — | |
| oracleScore=RAPS, Number of labeled data=802025.05 | 15.74 | 0.29 | — | — | — | — | — | — | — | — | — | |
| semicpScore=RAPS, Number of labeled data=802025.05 | 15.9 | 1.64 | — | — | — | — | — | — | — | — | — | |
| standardScore=RAPS, Number of labeled data=802025.05 | 16.57 | 2.7 | — | — | — | — | — | — | — | — | — | |
| APS+TTA-AvgAlpha=0.05, Augmentation Policy=Simple2025.05 | 18.862 | — | — | — | — | — | — | — | — | — | — | |
| APS+TTA-LearnedAlpha=0.05, Augmentation Policy=Simple2025.05 | 19.119 | — | — | — | — | — | — | — | — | — | — | |
| APSAlpha=0.05, Augmentation Policy=Expanded2025.05 | 19.82 | — | — | — | — | — | — | — | — | — | — | |
| APSAlpha=0.05, Augmentation Policy=Simple2025.05 | 19.82 | — | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=softmax, n_avg=202023.06 | 23 | 5.4 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=APS, n_avg=502023.06 | 25.6 | 2.5 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=APS, n_avg=752023.06 | 25.7 | 2.6 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=APS, n_avg=202023.06 | 25.8 | 2.6 | — | — | — | — | — | — | — | — | — | |
| Standard Conformal PredictionScore=APS, n_avg=102023.06 | 25.9 | 2.6 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=APS, n_avg=202023.06 | 27 | 2.6 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=APS, n_avg=502023.06 | 27.2 | 2.3 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=APS, n_avg=752023.06 | 27.3 | 2.2 | — | — | — | — | — | — | — | — | — | |
| Clustered Conformal PredictionScore=APS, n_avg=102023.06 | 29.6 | 3 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=RAPS, n_avg=202023.06 | 34.2 | 5.6 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=APS, n_avg=752023.06 | 35 | 2.9 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=APS, n_avg=502023.06 | 39.7 | 3.5 | — | — | — | — | — | — | — | — | — | |
| APS+TTA-AvgAlpha=0.01, Augmentation Policy=Expanded2025.05 | 68.714 | — | — | — | — | — | — | — | — | — | — | |
| APS+TTA-LearnedAlpha=0.01, Augmentation Policy=Expanded2025.05 | 69.009 | — | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=APS, n_avg=202023.06 | 76.9 | 5.5 | — | — | — | — | — | — | — | — | — | |
| APS+TTA-LearnedAlpha=0.01, Augmentation Policy=Simple2025.05 | 90.613 | — | — | — | — | — | — | — | — | — | — | |
| APS+TTA-AvgAlpha=0.01, Augmentation Policy=Simple2025.05 | 92.027 | — | — | — | — | — | — | — | — | — | — | |
| APSAlpha=0.01, Augmentation Policy=Expanded2025.05 | 98.493 | — | — | — | — | — | — | — | — | — | — | |
| APSAlpha=0.01, Augmentation Policy=Simple2025.05 | 98.493 | — | — | — | — | — | — | — | — | — | — | |
| oracleScore=APS, Number of labeled data=802025.05 | 166.15 | 0.3 | — | — | — | — | — | — | — | — | — | |
| semicpScore=APS, Number of labeled data=802025.05 | 166.98 | 1.13 | — | — | — | — | — | — | — | — | — | |
| standardScore=APS, Number of labeled data=802025.05 | 179.35 | 2.66 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=softmax, n_avg=102023.06 | 354.4 | 7.7 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=RAPS, n_avg=102023.06 | 361.9 | 7.7 | — | — | — | — | — | — | — | — | — | |
| Classwise Conformal PredictionScore=APS, n_avg=102023.06 | 394.2 | 7.6 | — | — | — | — | — | — | — | — | — | |
| APSEnergy variant=w/o Energy, Backbone=ResNet-502026.02 | — | — | 90 | 1.6 | 95 | 3.99 | 97.6 | 11.72 | 99 | 39.08 | — | |
| ClusteredBase Score Function=LAC, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 1.4 | |
| ClusteredBase Score Function=RAPS, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 2.46 | |
| ClusteredBase Score Function=SAPS, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 2.93 | |
| Energy-based APSEnergy variant=w/ Energy, Backbone=ResNet-502026.02 | — | — | 90 | 1.66 | 95 | 3.84 | 97.6 | 10.11 | 99 | 32.93 | — | |
| Energy-based RAPSEnergy variant=w/ Energy, Backbone=ResNet-502026.02 | — | — | 90 | 1.76 | 95 | 3.88 | 97.6 | 9.18 | 99 | 31.47 | — | |
| Energy-based SAPSEnergy variant=w/ Energy, Backbone=ResNet-502026.02 | — | — | 90 | 1.66 | 95 | 3.66 | 97.6 | 8.5 | 99 | 30.24 | — | |
| MS-CSBase Score Function=LAC, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 1.21 | |
| MS-CSBase Score Function=RAPS, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 1.44 | |
| MS-CSBase Score Function=SAPS, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 1.37 | |
| RAPSEnergy variant=w/o Energy, Backbone=ResNet-502026.02 | — | — | 90 | 1.77 | 95 | 4.22 | 97.6 | 10.56 | 99 | 37.01 | — | |
| SAPSEnergy variant=w/o Energy, Backbone=ResNet-502026.02 | — | — | 90 | 1.67 | 95 | 3.67 | 97.6 | 9.75 | 99 | 35.97 | — | |
| StandardBase Score Function=LAC, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 1.24 | |
| StandardBase Score Function=RAPS, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 2.48 | |
| StandardBase Score Function=SAPS, Backbone=ViT, alpha=0.12025.11 | — | — | — | — | — | — | — | — | — | — | 1.52 |