Conformal Prediction on CIFAR-100 (test)
1.6436Mean Prediction Set SizeTSCP
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| TSCPTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 1.6436 | 0.74 | — | 1.7924 | — | — | |
| APSTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 1.6514 | 0.72 | — | 2.9601 | — | — | |
| SCPTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 1.6958 | 0.8 | — | 2.3043 | — | — | |
| TSCPTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 1.8604 | 0.78 | — | 2.2327 | — | — | |
| TSCPTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 1.9905 | 0.66 | — | 6.2748 | — | — | |
| SCPTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 2.6004 | 0.7 | — | 3.5656 | — | — | |
| APSTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 2.6794 | 0.82 | — | 4.3883 | — | — | |
| LAC MS-CSScore Function=LAC, Method Variant=MS-CS, Backbone=RN50, alpha=0.052025.11 | 2.92 | — | — | — | — | 1.83 | |
| LAC MS-CSScore Function=LAC, Method Variant=MS-CS, Backbone=RN34, alpha=0.052025.11 | 2.94 | — | — | — | — | 1.87 | |
| SAPS MA-CSScore Function=SAPS, Method Variant=MA-CS, Backbone=RN50, alpha=0.052025.11 | 3.14 | — | — | — | — | 1.88 | |
| SAPS MS-CSScore Function=SAPS, Method Variant=MS-CS, Backbone=RN50, alpha=0.052025.11 | 3.14 | — | — | — | — | 1.97 | |
| LAC MA-CSScore Function=LAC, Method Variant=MA-CS, Backbone=RN50, alpha=0.052025.11 | 3.17 | — | — | — | — | 1.85 | |
| RAPS MS-CSScore Function=RAPS, Method Variant=MS-CS, Backbone=RN50, alpha=0.052025.11 | 3.17 | — | — | — | — | 1.95 | |
| APSTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 3.2094 | 0.74 | — | 5.3667 | — | — | |
| SAPS MA-CSScore Function=SAPS, Method Variant=MA-CS, Backbone=RN34, alpha=0.052025.11 | 3.32 | — | — | — | — | 1.26 | |
| SAPS MS-CSScore Function=SAPS, Method Variant=MS-CS, Backbone=RN34, alpha=0.052025.11 | 3.32 | — | — | — | — | 2.14 | |
| SAPS StandardScore Function=SAPS, Method Variant=Standard, Backbone=RN50, alpha=0.052025.11 | 3.45 | — | — | — | — | 2.33 | |
| RAPS MA-CSScore Function=RAPS, Method Variant=MA-CS, Backbone=RN50, alpha=0.052025.11 | 3.5 | — | — | — | — | 2.01 | |
| LAC MA-CSScore Function=LAC, Method Variant=MA-CS, Backbone=RN34, alpha=0.052025.11 | 3.51 | — | — | — | — | 1.92 | |
| SCPTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.09 | 3.5609 | 0.84 | — | 5.6643 | — | — | |
| SAPS StandardScore Function=SAPS, Method Variant=Standard, Backbone=RN34, alpha=0.052025.11 | 3.57 | — | — | — | — | 2.39 | |
| LAC ClusteredScore Function=LAC, Method Variant=Clustered, Backbone=RN34, alpha=0.052025.11 | 3.62 | — | — | — | — | 2.34 | |
| RAPS ClusteredScore Function=RAPS, Method Variant=Clustered, Backbone=RN50, alpha=0.052025.11 | 3.67 | — | — | — | — | 2.42 | |
| LAC StandardScore Function=LAC, Method Variant=Standard, Backbone=RN50, alpha=0.052025.11 | 3.68 | — | — | — | — | 2.27 | |
| LAC ClusteredScore Function=LAC, Method Variant=Clustered, Backbone=RN50, alpha=0.052025.11 | 3.7 | — | — | — | — | 2.28 | |
| RAPS MS-CSScore Function=RAPS, Method Variant=MS-CS, Backbone=RN34, alpha=0.052025.11 | 3.79 | — | — | — | — | 2.22 | |
| LAC StandardScore Function=LAC, Method Variant=Standard, Backbone=RN34, alpha=0.052025.11 | 3.82 | — | — | — | — | 2.41 | |
| RAPS StandardScore Function=RAPS, Method Variant=Standard, Backbone=RN50, alpha=0.052025.11 | 3.83 | — | — | — | — | 2.49 | |
| SAPS ClusteredScore Function=SAPS, Method Variant=Clustered, Backbone=RN50, alpha=0.052025.11 | 3.86 | — | — | — | — | 2.55 | |
| SAPS ClusteredScore Function=SAPS, Method Variant=Clustered, Backbone=RN34, alpha=0.052025.11 | 4.02 | — | — | — | — | 2.62 | |
| RAPS MA-CSScore Function=RAPS, Method Variant=MA-CS, Backbone=RN34, alpha=0.052025.11 | 4.52 | — | — | — | — | 2.29 | |
| RAPS ClusteredScore Function=RAPS, Method Variant=Clustered, Backbone=RN34, alpha=0.052025.11 | 5.62 | — | — | — | — | 3.32 | |
| RAPS StandardScore Function=RAPS, Method Variant=Standard, Backbone=RN34, alpha=0.052025.11 | 5.79 | — | — | — | — | 3.4 | |
| LAC AIRScore Function=LAC, Method Variant=AIR, Backbone=RN50, alpha=0.052025.11 | 6.8 | — | — | — | — | 1.36 | |
| LAC AIRScore Function=LAC, Method Variant=AIR, Backbone=RN34, alpha=0.052025.11 | 7.15 | — | — | — | — | 1.43 | |
| SAPS AIRScore Function=SAPS, Method Variant=AIR, Backbone=RN50, alpha=0.052025.11 | 7.55 | — | — | — | — | 1.51 | |
| SAPS AIRScore Function=SAPS, Method Variant=AIR, Backbone=RN34, alpha=0.052025.11 | 7.95 | — | — | — | — | 1.59 | |
| RAPS AIRScore Function=RAPS, Method Variant=AIR, Backbone=RN50, alpha=0.052025.11 | 9.75 | — | — | — | — | 1.95 | |
| Model-based Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 14.61 | — | — | — | — | — | |
| RAPS AIRScore Function=RAPS, Method Variant=AIR, Backbone=RN34, alpha=0.052025.11 | 15.15 | — | — | — | — | 3.03 | |
| DPI boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 17.41 | — | — | — | — | — | |
| DPI boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 17.55 | — | — | — | — | — | |
| CEScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 19.7 | — | — | — | — | — | |
| Model-based Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 21.68 | — | — | — | — | — | |
| CEScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 26.02 | — | — | — | — | — | |
| ConfTrScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 32.8 | — | — | — | — | — | |
| ConfTr_classScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 32.91 | — | — | — | — | — | |
| Simple Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 33.8 | — | — | — | — | — | |
| Simple Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 40.3 | — | — | — | — | — | |
| ConfTrScore function=APS, alpha=0.01, Setting=Centralized2024.05 | 40.58 | — | — | — | — | — | |
| ConfTr_classScore function=THR, alpha=0.01, Setting=Centralized2024.05 | 66.48 | — | — | — | — | — | |
| APSn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.96 | — | 6.2001 | 6.1049 | — | |
| APSn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.84 | — | 6.4607 | 6.4597 | — | |
| APSn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.94 | — | 6.5126 | 6.5079 | — | |
| Hard Pseudo-Calibrationsigma=0.32026.02 | — | 0.533 | 0.79 | — | — | — | |
| Hard Pseudo-Calibrationsigma=0.52026.02 | — | 0.458 | 0.81 | — | — | — | |
| Hard Pseudo-Calibrationsigma=0.72026.02 | — | 0.381 | 0.81 | — | — | — | |
| SCPn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.9 | — | 6.106 | 5.9406 | — | |
| SCPn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.86 | — | 6.4524 | 6.4242 | — | |
| SCPn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.94 | — | 6.5564 | 6.5492 | — | |
| Source Calibrationsigma=0.32026.02 | — | 0.755 | 2.46 | — | — | — | |
| Source Calibrationsigma=0.52026.02 | — | 0.68 | 2.79 | — | — | — | |
| Source Calibrationsigma=0.72026.02 | — | 0.596 | 2.92 | — | — | — | |
| Source-Tuned Pseudo-Calibrationsigma=0.32026.02 | — | 0.954 | 18.8 | — | — | — | |
| Source-Tuned Pseudo-Calibrationsigma=0.52026.02 | — | 0.937 | 31.95 | — | — | — | |
| Source-Tuned Pseudo-Calibrationsigma=0.72026.02 | — | 0.901 | 39.22 | — | — | — | |
| Target Calibration (Oracle)sigma=0.32026.02 | — | 0.813 | 3.54 | — | — | — | |
| Target Calibration (Oracle)sigma=0.52026.02 | — | 0.809 | 6.16 | — | — | — | |
| Target Calibration (Oracle)sigma=0.72026.02 | — | 0.797 | 9.83 | — | — | — | |
| TSCPn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.86 | — | 4.5361 | 4.3462 | — | |
| TSCPn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.88 | — | 6.5779 | 6.6439 | — | |
| TSCPn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09 | — | 0.92 | — | 6.6439 | 6.6439 | — |