Conformal Prediction on 5-class synthetic data Example 1, Easy Bin (r* > 0.5)
99.7CoverageAdaTS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AdaTSNumber of Calibration Samples=20002026.06 | 99.7 | 1.427 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 99.5 | 1.418 | |
| TrustscoreNumber of Calibration Samples=50002026.06 | 99.5 | 1.194 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 99.4 | 1.46 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 99.4 | 1.39 | |
| StandardNumber of Calibration Samples=20002026.06 | 99.4 | 1.155 | |
| TrustscoreNumber of Calibration Samples=20002026.06 | 99.4 | 1.24 | |
| StandardNumber of Calibration Samples=50002026.06 | 99.4 | 1.154 | |
| TrustscoreNumber of Calibration Samples=5002026.06 | 99.3 | 1.353 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 99.2 | 1.551 | |
| StandardNumber of Calibration Samples=5002026.06 | 99.2 | 1.163 | |
| StandardNumber of Calibration Samples=10002026.06 | 99.2 | 1.146 | |
| TrustscoreNumber of Calibration Samples=10002026.06 | 99.2 | 1.266 | |
| TSNumber of Calibration Samples=20002026.06 | 99 | 1.189 | |
| TSNumber of Calibration Samples=50002026.06 | 99 | 1.189 | |
| StandardNumber of Calibration Samples=2002026.06 | 98.8 | 1.198 | |
| TSNumber of Calibration Samples=5002026.06 | 98.8 | 1.193 | |
| TSNumber of Calibration Samples=10002026.06 | 98.8 | 1.173 | |
| TrustscoreNumber of Calibration Samples=2002026.06 | 98.6 | 1.581 | |
| TSNumber of Calibration Samples=2002026.06 | 98.4 | 1.238 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 97.7 | 1.39 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 95.5 | 1.956 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 95.3 | 1.664 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 95 | 1.25 | |
| RetrainNumber of Calibration Samples=20002026.06 | 94.5 | 2.159 | |
| RetrainNumber of Calibration Samples=10002026.06 | 94.2 | 2.377 | |
| RetrainNumber of Calibration Samples=50002026.06 | 93.8 | 1.52 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 93.6 | 1.545 | |
| RetrainNumber of Calibration Samples=5002026.06 | 93.5 | 2.628 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 92.9 | 1.287 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 92.4 | 1.417 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 92.4 | 1.28 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 92.3 | 1.306 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 92.3 | 1.222 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 92.1 | 1.41 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 91.8 | 1.309 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 91.3 | 1.311 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 90.9 | 1.26 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 90.6 | 1.194 | |
| RetrainNumber of Calibration Samples=2002026.06 | 90.4 | 3.774 | |
| ACP-MC-OracleNumber of Calibration Samples=5002026.06 | 90.3 | 0.903 | |
| ACP-MC-OracleNumber of Calibration Samples=20002026.06 | 90.2 | 0.902 | |
| ACP-MC-OracleNumber of Calibration Samples=50002026.06 | 90.2 | 0.902 | |
| ACP-MC-OracleNumber of Calibration Samples=10002026.06 | 90.1 | 0.901 | |
| ACP-MC-OracleNumber of Calibration Samples=2002026.06 | 90 | 0.9 |