Conformal Prediction on CIFAR 100
1.15Avg Prediction Set Sizeoracle
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| oracleScore=THR, Number of labeled data=802025.05 | 1.15 | 0.69 | — | |
| semicpScore=THR, Number of labeled data=802025.05 | 1.19 | 2.57 | — | |
| standardScore=THR, Number of labeled data=802025.05 | 1.21 | 2.68 | — | |
| DCO-WarmstartTarget coverage=0.802026.05 | 2.527 | — | 79.94 | |
| BQ/CRCTarget coverage=0.802026.05 | 2.7219 | — | 81.1 | |
| DCO-WarmstartTarget coverage=0.902026.05 | 5.3802 | — | 89.78 | |
| BQ/CRCTarget coverage=0.902026.05 | 5.8332 | — | 90.71 | |
| Standard Conformal PredictionScore=softmax, n_avg=102023.06 | 8.1 | 4 | — | |
| RAPS STANDARDScore=RAPS, n_avg=102023.06 | 8.6 | 4.7 | — | |
| Clustered Conformal PredictionScore=softmax, n_avg=102023.06 | 8.9 | 4.2 | — | |
| DCO-WarmstartTarget coverage=0.952026.05 | 10.0229 | — | 94.88 | |
| BQ/CRCTarget coverage=0.952026.05 | 10.732 | — | 95.47 | |
| semicpScore=RAPS, Number of labeled data=802025.05 | 10.92 | 0.95 | — | |
| oracleScore=RAPS, Number of labeled data=802025.05 | 10.93 | 0.67 | — | |
| standardScore=RAPS, Number of labeled data=802025.05 | 11.02 | 2.83 | — | |
| Standard Conformal PredictionScore=APS, n_avg=102023.06 | 11.2 | 3.7 | — | |
| DPIScoring Function=RAPS, Target Alpha=0.01, k_reg=1, lambda_reg=0.012024.05 | 17.25 | — | — | |
| DPIScoring Function=APS, Target Alpha=0.012024.05 | 17.41 | — | — | |
| MB FanoScoring Function=RAPS, Target Alpha=0.01, k_reg=1, lambda_reg=0.012024.05 | 20.97 | — | — | |
| MB FanoScoring Function=APS, Target Alpha=0.012024.05 | 21.68 | — | — | |
| CEScoring Function=APS, Target Alpha=0.012024.05 | 26.02 | — | — | |
| CEScoring Function=RAPS, Target Alpha=0.01, k_reg=1, lambda_reg=0.012024.05 | 26.02 | — | — | |
| ConfTrclassScoring Function=APS, Target Alpha=0.012024.05 | 32.91 | — | — | |
| ConfTrclassScoring Function=RAPS, Target Alpha=0.01, k_reg=1, lambda_reg=0.012024.05 | 32.94 | — | — | |
| FanoScoring Function=RAPS, Target Alpha=0.01, k_reg=1, lambda_reg=0.012024.05 | 33.75 | — | — | |
| FanoScoring Function=APS, Target Alpha=0.012024.05 | 33.8 | — | — | |
| semicpScore=APS, Number of labeled data=802025.05 | 38.56 | 0.75 | — | |
| oracleScore=APS, Number of labeled data=802025.05 | 38.57 | 0.67 | — | |
| standardScore=APS, Number of labeled data=802025.05 | 39.4 | 2.88 | — | |
| ConfTrScoring Function=RAPS, Target Alpha=0.01, k_reg=1, lambda_reg=0.012024.05 | 39.87 | — | — | |
| ConfTrScoring Function=APS, Target Alpha=0.012024.05 | 40.59 | — | — | |
| Classwise Conformal PredictionScore=softmax, n_avg=102023.06 | 47.2 | 7.6 | — |