Conformal Inference on Average across 15 datasets (test)
81.1Top-1 AccuracyLAC w/ Conf-OT
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| LAC w/ Conf-OTBackbone=MetaCLIP ViT-H/14, Alpha=0.102025.05 | 81.1 | 90 | 1.59 | 10.07 | |
| APS w/ Conf-OTBackbone=MetaCLIP ViT-H/14, Alpha=0.102025.05 | 81.1 | 89.9 | 3.47 | 6.6 | |
| RAPS w/ Conf-OTBackbone=MetaCLIP ViT-H/14, Alpha=0.102025.05 | 81.1 | 89.9 | 2.93 | 6.61 | |
| LAC w/ Conf-OTBackbone=MetaCLIP ViT-H/14, Alpha=0.052025.05 | 81.1 | 95 | 2.53 | 5.92 | |
| APS w/ Conf-OTBackbone=MetaCLIP ViT-H/14, Alpha=0.052025.05 | 81.1 | 94.9 | 6 | 4.55 | |
| RAPS w/ Conf-OTBackbone=MetaCLIP ViT-H/14, Alpha=0.052025.05 | 81.1 | 95 | 4.3 | 4.57 | |
| LACBackbone=MetaCLIP ViT-H/14, Alpha=0.102025.05 | 79.4 | 90 | 1.79 | 10.91 | |
| APSBackbone=MetaCLIP ViT-H/14, Alpha=0.102025.05 | 79.4 | 90 | 4.45 | 7.48 | |
| RAPSBackbone=MetaCLIP ViT-H/14, Alpha=0.102025.05 | 79.4 | 90 | 3.45 | 7.53 | |
| LACBackbone=MetaCLIP ViT-H/14, Alpha=0.052025.05 | 79.4 | 95 | 2.98 | 6.17 | |
| APSBackbone=MetaCLIP ViT-H/14, Alpha=0.052025.05 | 79.4 | 95 | 7.7 | 4.97 | |
| RAPSBackbone=MetaCLIP ViT-H/14, Alpha=0.052025.05 | 79.4 | 95 | 4.9 | 5.01 | |
| LAC w/ Conf-OTBackbone=CLIP ViT-L/14, Alpha=0.102025.05 | 74.7 | 90 | 2.5 | 9.97 | |
| APS w/ Conf-OTBackbone=CLIP ViT-L/14, Alpha=0.102025.05 | 74.7 | 90 | 4.87 | 6.98 | |
| RAPS w/ Conf-OTBackbone=CLIP ViT-L/14, Alpha=0.102025.05 | 74.7 | 90 | 4.13 | 7.01 | |
| LAC w/ Conf-OTBackbone=CLIP ViT-L/14, Alpha=0.052025.05 | 74.7 | 94.9 | 4.49 | 5.93 | |
| APS w/ Conf-OTBackbone=CLIP ViT-L/14, Alpha=0.052025.05 | 74.7 | 94.9 | 8.39 | 4.84 | |
| RAPS w/ Conf-OTBackbone=CLIP ViT-L/14, Alpha=0.052025.05 | 74.7 | 95 | 6.22 | 4.89 | |
| LACBackbone=CLIP ViT-L/14, Alpha=0.102025.05 | 72.6 | 90 | 2.93 | 10.34 | |
| APSBackbone=CLIP ViT-L/14, Alpha=0.102025.05 | 72.6 | 90 | 6.4 | 7.8 | |
| RAPSBackbone=CLIP ViT-L/14, Alpha=0.102025.05 | 72.6 | 90 | 4.93 | 7.87 | |
| LACBackbone=CLIP ViT-L/14, Alpha=0.052025.05 | 72.6 | 95 | 5.36 | 6.03 | |
| APSBackbone=CLIP ViT-L/14, Alpha=0.052025.05 | 72.6 | 94.9 | 11.27 | 5.21 | |
| RAPSBackbone=CLIP ViT-L/14, Alpha=0.052025.05 | 72.6 | 94.9 | 7.31 | 5.27 | |
| LAC w/ Conf-OTBackbone=MetaCLIP ViT-B/16, Alpha=0.102025.05 | 71.8 | 90 | 3.06 | 9.47 | |
| APS w/ Conf-OTBackbone=MetaCLIP ViT-B/16, Alpha=0.102025.05 | 71.8 | 90 | 5.58 | 7.12 | |
| RAPS w/ Conf-OTBackbone=MetaCLIP ViT-B/16, Alpha=0.102025.05 | 71.8 | 90 | 5.11 | 7.13 | |
| LAC w/ Conf-OTBackbone=MetaCLIP ViT-B/16, Alpha=0.052025.05 | 71.8 | 95 | 5.57 | 5.63 | |
| APS w/ Conf-OTBackbone=MetaCLIP ViT-B/16, Alpha=0.052025.05 | 71.8 | 95 | 9.8 | 4.85 | |
| RAPS w/ Conf-OTBackbone=MetaCLIP ViT-B/16, Alpha=0.052025.05 | 71.8 | 95 | 7.71 | 4.9 | |
| LACBackbone=MetaCLIP ViT-B/16, Alpha=0.102025.05 | 69.8 | 90 | 3.79 | 10.04 | |
| APSBackbone=MetaCLIP ViT-B/16, Alpha=0.102025.05 | 69.8 | 90 | 7.28 | 7.76 | |
| RAPSBackbone=MetaCLIP ViT-B/16, Alpha=0.102025.05 | 69.8 | 90 | 6.03 | 7.83 | |
| LACBackbone=MetaCLIP ViT-B/16, Alpha=0.052025.05 | 69.8 | 95 | 7.08 | 5.92 | |
| APSBackbone=MetaCLIP ViT-B/16, Alpha=0.052025.05 | 69.8 | 94.9 | 12.4 | 5.13 | |
| RAPSBackbone=MetaCLIP ViT-B/16, Alpha=0.052025.05 | 69.8 | 95 | 9.21 | 5.16 | |
| LAC w/ Conf-OTBackbone=CLIP ViT-B/32, Alpha=0.102025.05 | 61.3 | 89.9 | 6.63 | 9.39 | |
| APS w/ Conf-OTBackbone=CLIP ViT-B/32, Alpha=0.102025.05 | 61.3 | 90 | 10.69 | 7.59 | |
| RAPS w/ Conf-OTBackbone=CLIP ViT-B/32, Alpha=0.102025.05 | 61.3 | 89.9 | 9.25 | 7.66 | |
| LAC w/ Conf-OTBackbone=CLIP ViT-B/32, Alpha=0.052025.05 | 61.3 | 95 | 12.05 | 5.76 | |
| APS w/ Conf-OTBackbone=CLIP ViT-B/32, Alpha=0.052025.05 | 61.3 | 95 | 17.49 | 5.05 | |
| RAPS w/ Conf-OTBackbone=CLIP ViT-B/32, Alpha=0.052025.05 | 61.3 | 95 | 14.1 | 5.13 | |
| LAC w/ Conf-OTBackbone=CLIP ResNet-101, Alpha=0.102025.05 | 59.8 | 90 | 7.22 | 9.48 | |
| APS w/ Conf-OTBackbone=CLIP ResNet-101, Alpha=0.102025.05 | 59.8 | 90 | 11.4 | 7.74 | |
| RAPS w/ Conf-OTBackbone=CLIP ResNet-101, Alpha=0.102025.05 | 59.8 | 90 | 9.93 | 7.82 | |
| LAC w/ Conf-OTBackbone=CLIP ResNet-101, Alpha=0.052025.05 | 59.8 | 95 | 12.99 | 5.75 | |
| APS w/ Conf-OTBackbone=CLIP ResNet-101, Alpha=0.052025.05 | 59.8 | 95 | 18.69 | 5.15 | |
| RAPS w/ Conf-OTBackbone=CLIP ResNet-101, Alpha=0.052025.05 | 59.8 | 95 | 15.27 | 5.23 | |
| LACBackbone=CLIP ViT-B/32, Alpha=0.102025.05 | 58.7 | 90 | 8.2 | 10.4 | |
| APSBackbone=CLIP ViT-B/32, Alpha=0.102025.05 | 58.7 | 90.1 | 13.55 | 8.59 | |
| RAPSBackbone=CLIP ViT-B/32, Alpha=0.102025.05 | 58.7 | 90.1 | 11 | 8.7 | |
| LACBackbone=CLIP ViT-B/32, Alpha=0.052025.05 | 58.7 | 95 | 15.21 | 6.12 | |
| APSBackbone=CLIP ViT-B/32, Alpha=0.052025.05 | 58.7 | 95 | 22.76 | 5.54 | |
| RAPSBackbone=CLIP ViT-B/32, Alpha=0.052025.05 | 58.7 | 95 | 18.09 | 5.61 | |
| LACBackbone=CLIP ResNet-101, Alpha=0.102025.05 | 56.7 | 89.9 | 9.1 | 10.31 | |
| APSBackbone=CLIP ResNet-101, Alpha=0.102025.05 | 56.7 | 90 | 14.97 | 8.67 | |
| RAPSBackbone=CLIP ResNet-101, Alpha=0.102025.05 | 56.7 | 90.1 | 12.12 | 8.79 | |
| LACBackbone=CLIP ResNet-101, Alpha=0.052025.05 | 56.7 | 95 | 16.43 | 6.14 | |
| APSBackbone=CLIP ResNet-101, Alpha=0.052025.05 | 56.7 | 95 | 24.76 | 5.55 | |
| RAPSBackbone=CLIP ResNet-101, Alpha=0.052025.05 | 56.7 | 95 | 19.12 | 5.6 |