Conformal Prediction on Average across 15 datasets
95.7CoverageCPavg
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CPavgBackbone=MobileCLIP2-S2, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.7 | — | 6 | — | |
| CPBackbone=MobileCLIP2-S2, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.7 | — | 6.1 | — | |
| CPavgBackbone=MobileCLIP2-S3, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.7 | — | 4.2 | — | |
| CPBackbone=MobileCLIP2-B, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.7 | — | 5.2 | — | |
| CPavgBackbone=SigLIP2-SO400M/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.6 | — | 3.7 | — | |
| CPBackbone=MobileCLIP2-S3, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.6 | — | 4.4 | — | |
| CPavgBackbone=MobileCLIP2-S4, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.6 | — | 3.4 | — | |
| CPBackbone=MobileCLIP2-S4, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.6 | — | 3.4 | — | |
| CPavgBackbone=MobileCLIP2-B, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.6 | — | 4.8 | — | |
| CPBackbone=SigLIP2-B/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.5 | — | 5.3 | — | |
| CPBackbone=SigLIP2-SO400M/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.5 | — | 4 | — | |
| CPavgBackbone=SigLIP2-B/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.4 | — | 4.6 | — | |
| CPBackbone=SigLIP2-B/32, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.4 | — | 14.1 | — | |
| CPavgBackbone=SigLIP2-L/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.4 | — | 4.2 | — | |
| CPavgBackbone=SigLIP2-B/32, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.3 | — | 13.1 | — | |
| CPBackbone=SigLIP2-L/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.3 | — | 4.5 | — | |
| CPr-valueBackbone=MobileCLIP2-S3, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.3 | — | 4 | — | |
| CPr-valueBackbone=MobileCLIP2-S4, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.3 | — | 3.2 | — | |
| CPr-valueBackbone=SigLIP2-B/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.2 | — | 4 | — | |
| CPr-valueBackbone=SigLIP2-L/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.2 | — | 3.8 | — | |
| CPr-valueBackbone=MobileCLIP2-S2, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.2 | — | 5.9 | — | |
| CPr-valueBackbone=MobileCLIP2-B, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.2 | — | 4.7 | — | |
| CPr-valueBackbone=SigLIP2-B/32, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95.1 | — | 11.8 | — | |
| CPr-valueBackbone=SigLIP2-SO400M/16, Alpha (α)=0.05, Number of paraphrases=302026.05 | 95 | — | 3.3 | — | |
| Conf-OTNon-conformity score=LAC, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 66.7 | 4.4 | 9.48 | |
| APSNon-conformity score=APS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 63.8 | 9.87 | 8.39 | |
| TIM_KL (mu_K || u_K)Non-conformity score=APS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 64.7 | 7.24 | 9.32 | |
| TIM_KL (m || mu_K)Non-conformity score=APS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 65 | 7.82 | 9.38 | |
| RAPSNon-conformity score=RAPS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 63.8 | 8.12 | 8.5 | |
| TIM_KL (mu_K || u_K)Non-conformity score=RAPS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 64.7 | 7.18 | 9.32 | |
| TIM_KL (m || mu_K)Non-conformity score=RAPS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 65 | 7.68 | 9.42 | |
| Conf-OTNon-conformity score=RAPS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 90 | 66.7 | 6.68 | 7.48 | |
| LACNon-conformity score=LAC, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 89.9 | 63.8 | 5.52 | 10.37 | |
| TIM_KL (mu_K || u_K)Non-conformity score=LAC, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 89.9 | 64.7 | 8.3 | 10.41 | |
| Conf-OTNon-conformity score=APS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 89.9 | 66.7 | 7.64 | 7.44 | |
| TransCLIPNon-conformity score=RAPS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 89.9 | 65.1 | 7.17 | 10.2 | |
| TIM_KL (m || mu_K)Non-conformity score=LAC, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 89.8 | 65 | 7.73 | 10.89 | |
| TransCLIPNon-conformity score=LAC, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 89.2 | 65.1 | 5.76 | 11.02 | |
| TransCLIPNon-conformity score=APS, Backbone=CLIP ViT-B/16, Alpha=0.102025.05 | 89.2 | 65.1 | 8.27 | 11.5 |