Conformal Prediction on Example 2 5-class data (a=0.5, Hard Bin, r* <= 0.5) synthetic (test)
90.4CoverageACP-MC-Oracle
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACP-MC-OracleNumber of Calibration Samples=20002026.06 | 90.4 | 3.419 | |
| ACP-MC-OracleNumber of Calibration Samples=50002026.06 | 90.3 | 3.345 | |
| ACP-MC-OracleNumber of Calibration Samples=5002026.06 | 90.1 | 3.441 | |
| ACP-MC-OracleNumber of Calibration Samples=10002026.06 | 89.9 | 3.376 | |
| ACP-MC-OracleNumber of Calibration Samples=2002026.06 | 89.7 | 3.39 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 85.6 | 4.07 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 84.9 | 3.989 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 83.8 | 3.957 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 82.1 | 3.905 | |
| RetrainNumber of Calibration Samples=50002026.06 | 82.1 | 3.371 | |
| RetrainNumber of Calibration Samples=2002026.06 | 81.9 | 3.944 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 81.9 | 3.735 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 81.7 | 3.646 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 81.7 | 3.667 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 81.5 | 3.945 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 81.4 | 3.639 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 79.2 | 3.631 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 79 | 3.898 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 78.2 | 3.569 | |
| RetrainNumber of Calibration Samples=20002026.06 | 78.1 | 3.522 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 78 | 3.571 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 77.5 | 3.565 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 77.2 | 3.523 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 76.7 | 3.435 | |
| TrustscoreNumber of Calibration Samples=2002026.06 | 76.6 | 3.509 | |
| TSNumber of Calibration Samples=5002026.06 | 76.4 | 3.422 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 76.3 | 3.424 | |
| TSNumber of Calibration Samples=2002026.06 | 76.2 | 3.421 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 76.2 | 3.403 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 76.2 | 3.397 | |
| TSNumber of Calibration Samples=50002026.06 | 76.2 | 3.401 | |
| RetrainNumber of Calibration Samples=10002026.06 | 76.1 | 3.422 | |
| TSNumber of Calibration Samples=10002026.06 | 75.9 | 3.403 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 75.8 | 3.378 | |
| TSNumber of Calibration Samples=20002026.06 | 75.7 | 3.378 | |
| TrustscoreNumber of Calibration Samples=5002026.06 | 75.6 | 3.464 | |
| StandardNumber of Calibration Samples=5002026.06 | 74.6 | 3.383 | |
| StandardNumber of Calibration Samples=50002026.06 | 74.4 | 3.372 | |
| TrustscoreNumber of Calibration Samples=50002026.06 | 74.3 | 3.386 | |
| StandardNumber of Calibration Samples=2002026.06 | 74.1 | 3.369 | |
| TrustscoreNumber of Calibration Samples=10002026.06 | 74.1 | 3.377 | |
| StandardNumber of Calibration Samples=20002026.06 | 74.1 | 3.338 | |
| StandardNumber of Calibration Samples=10002026.06 | 73.9 | 3.366 | |
| TrustscoreNumber of Calibration Samples=20002026.06 | 73.9 | 3.343 | |
| RetrainNumber of Calibration Samples=5002026.06 | 73 | 3.325 |