List your company
ResearchFormerly WizWand

Conformal Prediction on CIFAR-100 (test)

1.6436Mean Prediction Set SizeTSCP
-0.94985616.55597234.061851.567628May 3, 2024Aug 6, 2024Nov 9, 2024Feb 12, 2025May 18, 2025Aug 21, 2025Nov 24, 2025
Updated 4mo ago

Evaluation results

MethodLinks
TSCPTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.091.64360.74—1.7924——
APSTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.091.65140.72—2.9601——
SCPTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.091.69580.8—2.3043——
TSCPTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.091.86040.78—2.2327——
TSCPTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.091.99050.66—6.2748——
SCPTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.092.60040.7—3.5656——
APSTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.092.67940.82—4.3883——
LAC MS-CSScore Function=LAC, Method Variant=MS-CS, Backbone=RN50, alpha=0.052025.112.92————1.83
LAC MS-CSScore Function=LAC, Method Variant=MS-CS, Backbone=RN34, alpha=0.052025.112.94————1.87
SAPS MA-CSScore Function=SAPS, Method Variant=MA-CS, Backbone=RN50, alpha=0.052025.113.14————1.88
SAPS MS-CSScore Function=SAPS, Method Variant=MS-CS, Backbone=RN50, alpha=0.052025.113.14————1.97
LAC MA-CSScore Function=LAC, Method Variant=MA-CS, Backbone=RN50, alpha=0.052025.113.17————1.85
RAPS MS-CSScore Function=RAPS, Method Variant=MS-CS, Backbone=RN50, alpha=0.052025.113.17————1.95
APSTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.093.20940.74—5.3667——
SAPS MA-CSScore Function=SAPS, Method Variant=MA-CS, Backbone=RN34, alpha=0.052025.113.32————1.26
SAPS MS-CSScore Function=SAPS, Method Variant=MS-CS, Backbone=RN34, alpha=0.052025.113.32————2.14
SAPS StandardScore Function=SAPS, Method Variant=Standard, Backbone=RN50, alpha=0.052025.113.45————2.33
RAPS MA-CSScore Function=RAPS, Method Variant=MA-CS, Backbone=RN50, alpha=0.052025.113.5————2.01
LAC MA-CSScore Function=LAC, Method Variant=MA-CS, Backbone=RN34, alpha=0.052025.113.51————1.92
SCPTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=ResNet202025.093.56090.84—5.6643——
SAPS StandardScore Function=SAPS, Method Variant=Standard, Backbone=RN34, alpha=0.052025.113.57————2.39
LAC ClusteredScore Function=LAC, Method Variant=Clustered, Backbone=RN34, alpha=0.052025.113.62————2.34
RAPS ClusteredScore Function=RAPS, Method Variant=Clustered, Backbone=RN50, alpha=0.052025.113.67————2.42
LAC StandardScore Function=LAC, Method Variant=Standard, Backbone=RN50, alpha=0.052025.113.68————2.27
LAC ClusteredScore Function=LAC, Method Variant=Clustered, Backbone=RN50, alpha=0.052025.113.7————2.28
RAPS MS-CSScore Function=RAPS, Method Variant=MS-CS, Backbone=RN34, alpha=0.052025.113.79————2.22
LAC StandardScore Function=LAC, Method Variant=Standard, Backbone=RN34, alpha=0.052025.113.82————2.41
RAPS StandardScore Function=RAPS, Method Variant=Standard, Backbone=RN50, alpha=0.052025.113.83————2.49
SAPS ClusteredScore Function=SAPS, Method Variant=Clustered, Backbone=RN50, alpha=0.052025.113.86————2.55
SAPS ClusteredScore Function=SAPS, Method Variant=Clustered, Backbone=RN34, alpha=0.052025.114.02————2.62
RAPS MA-CSScore Function=RAPS, Method Variant=MA-CS, Backbone=RN34, alpha=0.052025.114.52————2.29
RAPS ClusteredScore Function=RAPS, Method Variant=Clustered, Backbone=RN34, alpha=0.052025.115.62————3.32
RAPS StandardScore Function=RAPS, Method Variant=Standard, Backbone=RN34, alpha=0.052025.115.79————3.4
LAC AIRScore Function=LAC, Method Variant=AIR, Backbone=RN50, alpha=0.052025.116.8————1.36
LAC AIRScore Function=LAC, Method Variant=AIR, Backbone=RN34, alpha=0.052025.117.15————1.43
SAPS AIRScore Function=SAPS, Method Variant=AIR, Backbone=RN50, alpha=0.052025.117.55————1.51
SAPS AIRScore Function=SAPS, Method Variant=AIR, Backbone=RN34, alpha=0.052025.117.95————1.59
RAPS AIRScore Function=RAPS, Method Variant=AIR, Backbone=RN50, alpha=0.052025.119.75————1.95
Model-based Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.0514.61—————
RAPS AIRScore Function=RAPS, Method Variant=AIR, Backbone=RN34, alpha=0.052025.1115.15————3.03
DPI boundScore function=APS, alpha=0.01, Setting=Centralized2024.0517.41—————
DPI boundScore function=THR, alpha=0.01, Setting=Centralized2024.0517.55—————
CEScore function=THR, alpha=0.01, Setting=Centralized2024.0519.7—————
Model-based Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.0521.68—————
CEScore function=APS, alpha=0.01, Setting=Centralized2024.0526.02—————
ConfTrScore function=THR, alpha=0.01, Setting=Centralized2024.0532.8—————
ConfTr_classScore function=APS, alpha=0.01, Setting=Centralized2024.0532.91—————
Simple Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.0533.8—————
Simple Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.0540.3—————
ConfTrScore function=APS, alpha=0.01, Setting=Centralized2024.0540.58—————
ConfTr_classScore function=THR, alpha=0.01, Setting=Centralized2024.0566.48—————
APSn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.96—6.20016.1049—
APSn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.84—6.46076.4597—
APSn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.94—6.51266.5079—
Hard Pseudo-Calibrationsigma=0.32026.02—0.5330.79———
Hard Pseudo-Calibrationsigma=0.52026.02—0.4580.81———
Hard Pseudo-Calibrationsigma=0.72026.02—0.3810.81———
SCPn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.9—6.1065.9406—
SCPn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.86—6.45246.4242—
SCPn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.94—6.55646.5492—
Source Calibrationsigma=0.32026.02—0.7552.46———
Source Calibrationsigma=0.52026.02—0.682.79———
Source Calibrationsigma=0.72026.02—0.5962.92———
Source-Tuned Pseudo-Calibrationsigma=0.32026.02—0.95418.8———
Source-Tuned Pseudo-Calibrationsigma=0.52026.02—0.93731.95———
Source-Tuned Pseudo-Calibrationsigma=0.72026.02—0.90139.22———
Target Calibration (Oracle)sigma=0.32026.02—0.8133.54———
Target Calibration (Oracle)sigma=0.52026.02—0.8096.16———
Target Calibration (Oracle)sigma=0.72026.02—0.7979.83———
TSCPn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.86—4.53614.3462—
TSCPn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.88—6.57796.6439—
TSCPn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=ResNet202025.09—0.92—6.64396.6439—