Conformal Prediction on 5-class synthetic data (Example 1) Easy Bin (r* > 0.5) 1.0 (test)
99.7CoverageAdaTS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AdaTSNumber of Calibration Samples=50002026.06 | 99.7 | 1.398 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 99.5 | 1.482 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 99.5 | 1.415 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 99.5 | 1.422 | |
| StandardNumber of Calibration Samples=50002026.06 | 99.4 | 1.144 | |
| StandardNumber of Calibration Samples=10002026.06 | 99.3 | 1.142 | |
| StandardNumber of Calibration Samples=20002026.06 | 99.3 | 1.147 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 99.2 | 1.652 | |
| StandardNumber of Calibration Samples=2002026.06 | 99.1 | 1.203 | |
| StandardNumber of Calibration Samples=3002026.06 | 99.1 | 1.164 | |
| StandardNumber of Calibration Samples=5002026.06 | 99.1 | 1.163 | |
| AdaTSNumber of Calibration Samples=3002026.06 | 98.9 | 1.557 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 97.7 | 1.39 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 96.2 | 1.89 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 95.7 | 1.683 | |
| AACP-MCNumber of Calibration Samples=2002026.06 | 95.2 | 1.568 | |
| ACP-ACCNumber of Calibration Samples=3002026.06 | 95 | 1.647 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 95 | 1.25 | |
| RetrainNumber of Calibration Samples=20002026.06 | 94.7 | 2.001 | |
| RetrainNumber of Calibration Samples=10002026.06 | 94.5 | 2.39 | |
| AACP-ACCNumber of Calibration Samples=5002026.06 | 94.4 | 1.939 | |
| ACP-MCNumber of Calibration Samples=3002026.06 | 94.2 | 1.525 | |
| RetrainNumber of Calibration Samples=5002026.06 | 94.2 | 2.632 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 93.8 | 1.607 | |
| AACP-MCNumber of Calibration Samples=5002026.06 | 93.6 | 1.48 | |
| RetrainNumber of Calibration Samples=50002026.06 | 93.6 | 1.441 | |
| AACP-MCNumber of Calibration Samples=3002026.06 | 93.4 | 1.553 | |
| AACP-ACCNumber of Calibration Samples=3002026.06 | 93.1 | 2.771 | |
| RetrainNumber of Calibration Samples=3002026.06 | 93.1 | 3.248 | |
| AACP-ACCNumber of Calibration Samples=10002026.06 | 93 | 1.486 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 92.9 | 1.287 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 92.7 | 1.437 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 92.5 | 1.379 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 92.4 | 1.28 | |
| AACP-AECNumber of Calibration Samples=50002026.06 | 92.3 | 1.351 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 92.3 | 1.222 | |
| AACP-AECNumber of Calibration Samples=20002026.06 | 92.2 | 1.597 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 92 | 1.194 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 91.8 | 1.29 | |
| AACP-MCNumber of Calibration Samples=10002026.06 | 91.7 | 1.294 | |
| AACP-ACCNumber of Calibration Samples=20002026.06 | 91.7 | 1.387 | |
| RetrainNumber of Calibration Samples=2002026.06 | 91.6 | 3.858 | |
| AACP-AECNumber of Calibration Samples=5002026.06 | 91.6 | 3.695 | |
| AACP-AECNumber of Calibration Samples=10002026.06 | 91.6 | 2.594 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 91.5 | 1.314 | |
| AACP-ACCNumber of Calibration Samples=50002026.06 | 91.4 | 1.211 | |
| AACP-ACCNumber of Calibration Samples=2002026.06 | 91.3 | 3.737 | |
| AACP-AECNumber of Calibration Samples=2002026.06 | 91.2 | 4.405 | |
| AACP-MCNumber of Calibration Samples=20002026.06 | 90.9 | 1.246 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 90.9 | 1.246 | |
| AACP-MCNumber of Calibration Samples=50002026.06 | 90.8 | 1.204 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 90.8 | 1.205 |