Conformal Prediction on CIFAR-10-C (Marginal Coverage & Quantile Size Analysis)
92.9Marginal CoverageACP-ACC
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ACP-ACCCalibration Size=2002026.06 | 92.9 | 3.843 | 87.4 | 8.302 | 95.8 | 7.798 | 93.2 | 2.809 | |
| ACP-AECCalibration Size=2002026.06 | 91.5 | 2.801 | 78 | 7.078 | 92.7 | 6.374 | 92.8 | 1.844 | |
| ACP-ACCCalibration Size=3002026.06 | 91.2 | 3.529 | 85.2 | 7.843 | 95.2 | 7.388 | 91.4 | 2.522 | |
| RetrainCalibration Size=2002026.06 | 91.1 | 9.136 | 90.6 | 9.156 | 91.6 | 9.141 | 91 | 9.133 | |
| AdaTSCalibration Size=2002026.06 | 90.9 | 2.588 | 57.3 | 4.587 | 91.4 | 4.659 | 94.4 | 2.082 | |
| ACP-AECCalibration Size=3002026.06 | 90.9 | 2.736 | 72.4 | 6.271 | 93.6 | 6.337 | 92.5 | 1.85 | |
| ACP-MCCalibration Size=2002026.06 | 90.8 | 3.9 | 84.9 | 8.36 | 90.4 | 7.658 | 91.5 | 2.89 | |
| ACP-ACCCalibration Size=5002026.06 | 90.6 | 3.055 | 76.8 | 7.097 | 93.6 | 6.65 | 91.7 | 2.118 | |
| ACP-ACCCalibration Size=10002026.06 | 90.4 | 2.807 | 77.9 | 7.152 | 93.2 | 6.351 | 91.4 | 1.845 | |
| StandardCalibration Size=2002026.06 | 90.2 | 1.968 | 48.9 | 3.712 | 87.7 | 3.926 | 95 | 1.505 | |
| ACP-AECCalibration Size=5002026.06 | 90.2 | 2.636 | 67.4 | 5.982 | 92.2 | 6.002 | 92.4 | 1.807 | |
| ACP-MCCalibration Size=10002026.06 | 90.2 | 3.684 | 86.4 | 8.315 | 94 | 7.623 | 90.1 | 2.632 | |
| ACP-AECCalibration Size=20002026.06 | 90.2 | 2.709 | 70.4 | 6.192 | 94.1 | 6.376 | 91.8 | 1.818 | |
| ACP-ACCCalibration Size=20002026.06 | 90.1 | 2.701 | 80.3 | 7.331 | 93.1 | 6.247 | 90.8 | 1.701 | |
| ACP-MCCalibration Size=3002026.06 | 90 | 3.746 | 84.8 | 8.152 | 90.9 | 7.502 | 90.5 | 2.745 | |
| AdaTSCalibration Size=3002026.06 | 90 | 2.404 | 52.2 | 4.104 | 90.7 | 4.431 | 93.9 | 1.936 | |
| RetrainCalibration Size=3002026.06 | 90 | 7.754 | 77.4 | 6.816 | 88.1 | 7.828 | 91.7 | 7.842 | |
| RetrainCalibration Size=5002026.06 | 90 | 7.268 | 80.2 | 6.891 | 88.4 | 7.331 | 91.3 | 7.3 | |
| StandardCalibration Size=5002026.06 | 90 | 1.943 | 44.3 | 3.433 | 88.1 | 3.904 | 95.2 | 1.512 | |
| ACP-AECCalibration Size=10002026.06 | 90 | 2.698 | 69.4 | 6.208 | 92.8 | 6.273 | 91.8 | 1.823 | |
| RetrainCalibration Size=10002026.06 | 90 | 6.392 | 79.5 | 6.624 | 85.5 | 6.585 | 91.7 | 6.338 | |
| StandardCalibration Size=10002026.06 | 90 | 1.924 | 45.1 | 3.49 | 87.5 | 3.859 | 95.1 | 1.484 | |
| ACP-MCCalibration Size=20002026.06 | 90 | 3.526 | 88.8 | 8.28 | 94.4 | 7.496 | 89.5 | 2.453 | |
| RetrainCalibration Size=20002026.06 | 90 | 5.591 | 81 | 6.701 | 85.3 | 6.053 | 91.7 | 5.406 | |
| StandardCalibration Size=20002026.06 | 90 | 1.929 | 45.8 | 3.549 | 88.6 | 3.893 | 95 | 1.478 | |
| StandardCalibration Size=3002026.06 | 89.9 | 1.917 | 45.8 | 3.459 | 88 | 3.901 | 94.9 | 1.47 | |
| AdaTSCalibration Size=5002026.06 | 89.9 | 2.425 | 50.9 | 4.152 | 90.9 | 4.403 | 94 | 1.968 | |
| AdaTSCalibration Size=10002026.06 | 89.9 | 2.396 | 51.2 | 4.184 | 89.9 | 4.363 | 93.9 | 1.927 | |
| AdaTSCalibration Size=20002026.06 | 89.9 | 2.365 | 52.6 | 4.182 | 90.9 | 4.346 | 93.9 | 1.891 | |
| ACP-MCCalibration Size=5002026.06 | 89.8 | 3.738 | 85.9 | 8.227 | 91.1 | 7.488 | 90 | 2.724 |