Conformal Prediction on MNIST (test)
100CoverageSCP
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SCPn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 100 | — | — | 3.2045 | 3.1923 | |
| TSCPn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 92 | — | — | 1.454 | 1.3801 | |
| Source-Tuned Pseudo-Calibrationsigma=0.72026.02 | 90.3 | — | 0.93 | — | — | |
| SCPn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 90 | — | — | 2.8162 | 2.7429 | |
| TSCPn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 90 | — | — | 1.4961 | 1.2871 | |
| Target Calibration (Oracle)sigma=0.72026.02 | 89.2 | — | 0.91 | — | — | |
| APSn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 88 | — | — | 2.8314 | 2.8313 | |
| SCPn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 88 | — | — | 3.1161 | 3.086 | |
| APSn (transductive size)=6, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 88 | — | — | 3.1037 | 3.1133 | |
| Hard Pseudo-Calibrationsigma=0.72026.02 | 87.1 | — | 0.88 | — | — | |
| APSn (transductive size)=9, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 86 | — | — | 3.177 | 3.1669 | |
| SCPTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 84 | 0 | — | -0.0615 | — | |
| TSCPn (transductive size)=3, alpha=0.1, noise level (1-epsilon)=0.9, Backbone=LeNet52025.09 | 84 | — | — | 1.4445 | 1.4641 | |
| Source Calibrationsigma=0.72026.02 | 83.7 | — | 0.84 | — | — | |
| Target Calibration (Oracle)sigma=1.62026.02 | 83.4 | — | 3.32 | — | — | |
| Target Calibration (Oracle)sigma=22026.02 | 81.3 | — | 5.45 | — | — | |
| APSTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 74 | 0.1469 | — | 1.9468 | — | |
| TSCPTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 74 | 0 | — | -0.0597 | — | |
| TSCPTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 74 | 0 | — | 0.0942 | — | |
| Source-Tuned Pseudo-Calibrationsigma=1.62026.02 | 71.8 | — | 1.87 | — | — | |
| SCPTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 70 | 0 | — | -0.0477 | — | |
| APSTransductive size (n)=6, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 70 | 0 | — | 0.3496 | — | |
| SCPTransductive size (n)=9, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 66 | 0 | — | 2.0215 | — | |
| APSTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 64 | 0 | — | -0.0615 | — | |
| TSCPTransductive size (n)=3, α=0.3, Noise level (1-ϵ)=0.98, Backbone=LeNet52025.09 | 64 | 0 | — | -0.2146 | — | |
| Hard Pseudo-Calibrationsigma=1.62026.02 | 53.9 | — | 0.83 | — | — | |
| Source Calibrationsigma=1.62026.02 | 44 | — | 0.55 | — | — | |
| Source-Tuned Pseudo-Calibrationsigma=22026.02 | 43.7 | — | 1.22 | — | — | |
| Hard Pseudo-Calibrationsigma=22026.02 | 34.7 | — | 0.82 | — | — | |
| Source Calibrationsigma=22026.02 | 27.6 | — | 0.5 | — | — | |
| CEScore function=THR, alpha=0.01, Setting=Centralized2024.05 | — | 2.29 | — | — | — | |
| CEScore function=APS, alpha=0.01, Setting=Centralized2024.05 | — | 2.5 | — | — | — | |
| ConfTrScore function=THR, alpha=0.01, Setting=Centralized2024.05 | — | 6.28 | — | — | — | |
| ConfTrScore function=APS, alpha=0.01, Setting=Centralized2024.05 | — | 2.1 | — | — | — | |
| ConfTr_classScore function=THR, alpha=0.01, Setting=Centralized2024.05 | — | 2.09 | — | — | — | |
| ConfTr_classScore function=APS, alpha=0.01, Setting=Centralized2024.05 | — | 2.13 | — | — | — | |
| DPI boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | — | 2.24 | — | — | — | |
| DPI boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | — | 2.64 | — | — | — | |
| Model-based Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | — | 2.24 | — | — | — | |
| Model-based Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | — | 2.49 | — | — | — | |
| Simple Fano boundScore function=THR, alpha=0.01, Setting=Centralized2024.05 | — | 2.09 | — | — | — | |
| Simple Fano boundScore function=APS, alpha=0.01, Setting=Centralized2024.05 | — | 2.12 | — | — | — |