Conformal Prediction on 5-class synthetic data Example 2 covariate shift a=0.5 Hard Bin (r* <= 0.5)
85.6CoverageACP-ACC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACP-ACCNumber of Calibration Samples=2002026.06 | 85.6 | 4.07 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 84.9 | 3.989 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 83.8 | 3.957 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 82.1 | 3.905 | |
| RetrainNumber of Calibration Samples=50002026.06 | 82.1 | 3.371 | |
| RetrainNumber of Calibration Samples=2002026.06 | 81.9 | 3.944 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 81.9 | 3.735 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 81.7 | 3.646 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 81.7 | 3.667 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 81.5 | 3.945 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 81.4 | 3.639 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 79.2 | 3.631 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 79 | 3.898 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 78.2 | 3.569 | |
| RetrainNumber of Calibration Samples=20002026.06 | 78.1 | 3.522 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 78 | 3.571 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 77.5 | 3.565 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 77.2 | 3.523 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 76.7 | 3.435 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 76.3 | 3.424 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 76.2 | 3.403 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 76.2 | 3.397 | |
| RetrainNumber of Calibration Samples=10002026.06 | 76.1 | 3.422 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 75.8 | 3.378 | |
| StandardNumber of Calibration Samples=5002026.06 | 74.6 | 3.383 | |
| StandardNumber of Calibration Samples=50002026.06 | 74.4 | 3.372 | |
| StandardNumber of Calibration Samples=2002026.06 | 74.1 | 3.369 | |
| StandardNumber of Calibration Samples=20002026.06 | 74.1 | 3.338 | |
| StandardNumber of Calibration Samples=10002026.06 | 73.9 | 3.366 | |
| RetrainNumber of Calibration Samples=5002026.06 | 73 | 3.325 |