Conformal Prediction on 5-class synthetic data Example 1 Hard Bin r* ≤ 0.5
93.4CoverageACP-ACC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACP-ACCNumber of Calibration Samples=20002026.06 | 93.4 | 4.601 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 92.8 | 4.498 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 91.7 | 4.598 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 91.7 | 4.598 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 91.6 | 4.476 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 91.3 | 4.59 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 91.3 | 4.59 | |
| AACP-AECNumber of Calibration Samples=50002026.06 | 91.2 | 4.58 | |
| AACP-AECNumber of Calibration Samples=20002026.06 | 91 | 4.547 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 90.7 | 4.413 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 90.6 | 4.355 | |
| ACP-MC-OracleNumber of Calibration Samples=5002026.06 | 90.4 | 3 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 90.2 | 4.324 | |
| ACP-MC-OracleNumber of Calibration Samples=50002026.06 | 90.2 | 2.959 | |
| ACP-MC-OracleNumber of Calibration Samples=2002026.06 | 90 | 2.977 | |
| ACP-MC-OracleNumber of Calibration Samples=10002026.06 | 89.8 | 2.986 | |
| ACP-MC-OracleNumber of Calibration Samples=20002026.06 | 89.8 | 2.958 | |
| AACP-ACCNumber of Calibration Samples=50002026.06 | 89.1 | 4.286 | |
| AACP-AECNumber of Calibration Samples=2002026.06 | 88.4 | 4.412 | |
| AACP-ACCNumber of Calibration Samples=10002026.06 | 88.2 | 4.085 | |
| AACP-ACCNumber of Calibration Samples=20002026.06 | 87.5 | 4.12 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 87 | 4.014 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 86.8 | 3.996 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 86.6 | 4.032 | |
| AACP-MCNumber of Calibration Samples=20002026.06 | 86.4 | 3.981 | |
| AACP-MCNumber of Calibration Samples=50002026.06 | 86.4 | 4.024 | |
| AACP-AECNumber of Calibration Samples=5002026.06 | 86.2 | 4.196 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 86.2 | 3.98 | |
| AACP-AECNumber of Calibration Samples=10002026.06 | 85.4 | 4.182 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 84.9 | 3.938 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 84.8 | 4.173 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 84.8 | 4.173 | |
| AACP-MCNumber of Calibration Samples=10002026.06 | 84.7 | 3.894 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 84.3 | 3.873 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 84 | 3.871 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 83.9 | 3.833 | |
| RetrainNumber of Calibration Samples=2002026.06 | 82.8 | 3.903 | |
| AACP-MCNumber of Calibration Samples=5002026.06 | 82.3 | 3.768 | |
| AACP-ACCNumber of Calibration Samples=2002026.06 | 81.8 | 3.857 | |
| AACP-ACCNumber of Calibration Samples=5002026.06 | 81.3 | 3.508 | |
| RetrainNumber of Calibration Samples=2002026.06 | 81.1 | 3.8 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 78.2 | 3.538 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 77.7 | 3.547 | |
| RetrainNumber of Calibration Samples=3002026.06 | 76.5 | 3.304 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 76.4 | 3.565 | |
| AACP-ACCNumber of Calibration Samples=3002026.06 | 76.3 | 3.34 | |
| ACP-ACCNumber of Calibration Samples=3002026.06 | 74.7 | 3.381 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 74.1 | 3.447 | |
| RetrainNumber of Calibration Samples=50002026.06 | 73.2 | 2.745 | |
| RetrainNumber of Calibration Samples=50002026.06 | 72.8 | 2.671 | |
| AACP-MCNumber of Calibration Samples=3002026.06 | 72.6 | 3.376 | |
| RetrainNumber of Calibration Samples=5002026.06 | 71.2 | 2.715 | |
| RetrainNumber of Calibration Samples=5002026.06 | 70 | 2.701 | |
| ACP-MCNumber of Calibration Samples=3002026.06 | 69.8 | 3.263 | |
| RetrainNumber of Calibration Samples=20002026.06 | 69.7 | 2.71 | |
| RetrainNumber of Calibration Samples=10002026.06 | 69.1 | 2.612 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 68.7 | 3.137 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 68.7 | 3.137 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 68.4 | 3.162 | |
| RetrainNumber of Calibration Samples=20002026.06 | 67.8 | 2.546 | |
| RetrainNumber of Calibration Samples=10002026.06 | 67.6 | 2.605 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 66.8 | 3.068 | |
| AACP-MCNumber of Calibration Samples=2002026.06 | 65.7 | 3.259 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 56.2 | 2.224 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 56.2 | 2.224 | |
| TrustscoreNumber of Calibration Samples=2002026.06 | 52.2 | 2.154 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 51.8 | 1.754 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 48.4 | 1.655 | |
| TSNumber of Calibration Samples=20002026.06 | 48.2 | 1.471 | |
| TrustscoreNumber of Calibration Samples=5002026.06 | 48 | 1.765 | |
| AdaTSNumber of Calibration Samples=3002026.06 | 47.9 | 1.654 | |
| TSNumber of Calibration Samples=50002026.06 | 47.7 | 1.466 | |
| TSNumber of Calibration Samples=2002026.06 | 47.6 | 1.495 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 47.2 | 1.571 | |
| TrustscoreNumber of Calibration Samples=20002026.06 | 47 | 1.539 | |
| TSNumber of Calibration Samples=10002026.06 | 46.7 | 1.449 | |
| TSNumber of Calibration Samples=5002026.06 | 46.6 | 1.458 | |
| StandardNumber of Calibration Samples=20002026.06 | 46.5 | 1.441 | |
| StandardNumber of Calibration Samples=2002026.06 | 46.3 | 1.458 | |
| StandardNumber of Calibration Samples=5002026.06 | 46.2 | 1.44 | |
| TrustscoreNumber of Calibration Samples=10002026.06 | 46.1 | 1.612 | |
| StandardNumber of Calibration Samples=50002026.06 | 46.1 | 1.437 | |
| StandardNumber of Calibration Samples=20002026.06 | 46.1 | 1.434 | |
| TrustscoreNumber of Calibration Samples=50002026.06 | 46 | 1.459 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 45.5 | 1.533 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 45.4 | 1.559 | |
| StandardNumber of Calibration Samples=50002026.06 | 45.4 | 1.424 | |
| StandardNumber of Calibration Samples=10002026.06 | 45.3 | 1.419 | |
| StandardNumber of Calibration Samples=2002026.06 | 45.1 | 1.449 | |
| StandardNumber of Calibration Samples=5002026.06 | 44.9 | 1.427 | |
| StandardNumber of Calibration Samples=3002026.06 | 44.9 | 1.432 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 44.9 | 1.535 | |
| StandardNumber of Calibration Samples=10002026.06 | 44.8 | 1.417 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 44.6 | 1.526 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 44.3 | 1.52 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 43.4 | 1.51 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 43.3 | 1.505 |