Conformal Prediction on synthetic data 5-class Example 2 covariate shift a=0.5 Easy Bin (r* > 0.5)
96.6CoverageStandard
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| StandardNumber of Calibration Samples=2002026.06 | 96.6 | 1.24 | |
| StandardNumber of Calibration Samples=5002026.06 | 96.6 | 1.231 | |
| StandardNumber of Calibration Samples=10002026.06 | 96.6 | 1.241 | |
| StandardNumber of Calibration Samples=50002026.06 | 96.6 | 1.259 | |
| StandardNumber of Calibration Samples=20002026.06 | 96.5 | 1.225 | |
| RetrainNumber of Calibration Samples=10002026.06 | 96 | 3.231 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 95.9 | 1.359 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 95.8 | 1.343 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 95.8 | 1.356 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 95.8 | 1.365 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 95.7 | 1.264 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 95.6 | 1.325 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 95.2 | 1.232 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 95 | 1.226 | |
| RetrainNumber of Calibration Samples=20002026.06 | 94.8 | 2.881 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 94.6 | 1.576 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 94.5 | 1.659 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 94.5 | 1.2 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 94.3 | 1.199 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 94 | 1.386 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 93.9 | 1.339 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 93.7 | 1.276 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 93.4 | 1.547 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 93.4 | 1.271 | |
| RetrainNumber of Calibration Samples=50002026.06 | 93.2 | 1.552 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 92.8 | 1.468 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 92.6 | 1.407 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 92.2 | 1.344 | |
| RetrainNumber of Calibration Samples=5002026.06 | 91.8 | 3.256 | |
| RetrainNumber of Calibration Samples=2002026.06 | 88.5 | 3.911 |