Conformal Prediction on 5-class synthetic data Example 1 Marginal
92.3CoverageACP-AEC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACP-AECNumber of Calibration Samples=20002026.06 | 92.3 | 1.895 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 92.2 | 1.827 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 92 | 1.906 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 92 | 1.854 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 91.9 | 1.968 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 91.8 | 1.937 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 91.7 | 2.215 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 91.4 | 1.804 | |
| TrustscoreNumber of Calibration Samples=2002026.06 | 90.5 | 1.681 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 90.3 | 1.914 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 90.3 | 1.57 | |
| ACP-MC-OracleNumber of Calibration Samples=5002026.06 | 90.3 | 1.267 | |
| TrustscoreNumber of Calibration Samples=5002026.06 | 90.3 | 1.426 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 90.2 | 1.537 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 90.2 | 1.563 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 90.2 | 1.754 | |
| ACP-MC-OracleNumber of Calibration Samples=20002026.06 | 90.2 | 1.264 | |
| ACP-MC-OracleNumber of Calibration Samples=50002026.06 | 90.2 | 1.264 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 90.1 | 1.764 | |
| ACP-MC-OracleNumber of Calibration Samples=10002026.06 | 90.1 | 1.265 | |
| RetrainNumber of Calibration Samples=20002026.06 | 90.1 | 2.258 | |
| TrustscoreNumber of Calibration Samples=20002026.06 | 90.1 | 1.295 | |
| RetrainNumber of Calibration Samples=50002026.06 | 90.1 | 1.736 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 90 | 1.445 | |
| StandardNumber of Calibration Samples=20002026.06 | 90 | 1.208 | |
| TSNumber of Calibration Samples=20002026.06 | 90 | 1.241 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 90 | 1.705 | |
| TrustscoreNumber of Calibration Samples=50002026.06 | 90 | 1.243 | |
| ACP-MC-OracleNumber of Calibration Samples=2002026.06 | 89.9 | 1.262 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 89.9 | 1.477 | |
| TrustscoreNumber of Calibration Samples=10002026.06 | 89.9 | 1.327 | |
| StandardNumber of Calibration Samples=50002026.06 | 89.9 | 1.206 | |
| TSNumber of Calibration Samples=50002026.06 | 89.9 | 1.24 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 89.8 | 1.791 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 89.8 | 1.437 | |
| StandardNumber of Calibration Samples=5002026.06 | 89.7 | 1.21 | |
| TSNumber of Calibration Samples=5002026.06 | 89.7 | 1.24 | |
| StandardNumber of Calibration Samples=10002026.06 | 89.7 | 1.194 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 89.6 | 1.41 | |
| TSNumber of Calibration Samples=10002026.06 | 89.6 | 1.222 | |
| TSNumber of Calibration Samples=2002026.06 | 89.5 | 1.284 | |
| RetrainNumber of Calibration Samples=10002026.06 | 89.5 | 2.418 | |
| StandardNumber of Calibration Samples=2002026.06 | 89.4 | 1.243 | |
| RetrainNumber of Calibration Samples=5002026.06 | 89.4 | 2.641 | |
| RetrainNumber of Calibration Samples=2002026.06 | 88.8 | 3.779 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 0.927 | 2.186 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 0.923 | 1.895 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 0.922 | 1.922 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 0.922 | 1.827 | |
| AACP-ACCNumber of Calibration Samples=10002026.06 | 0.921 | 1.95 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 0.921 | 1.916 | |
| AACP-AECNumber of Calibration Samples=50002026.06 | 0.921 | 1.926 | |
| AACP-ACCNumber of Calibration Samples=5002026.06 | 0.92 | 2.222 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 0.92 | 2.02 | |
| AACP-AECNumber of Calibration Samples=20002026.06 | 0.92 | 2.145 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 0.92 | 1.762 | |
| AACP-MCNumber of Calibration Samples=5002026.06 | 0.915 | 1.905 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 0.914 | 1.804 | |
| ACP-ACCNumber of Calibration Samples=3002026.06 | 0.912 | 1.949 | |
| AACP-ACCNumber of Calibration Samples=20002026.06 | 0.91 | 1.883 | |
| AACP-ACCNumber of Calibration Samples=50002026.06 | 0.91 | 1.748 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 0.909 | 1.953 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 0.909 | 1.67 | |
| AACP-MCNumber of Calibration Samples=2002026.06 | 0.907 | 1.828 | |
| AACP-AECNumber of Calibration Samples=2002026.06 | 0.906 | 4.407 | |
| AACP-AECNumber of Calibration Samples=5002026.06 | 0.906 | 3.776 | |
| AACP-AECNumber of Calibration Samples=10002026.06 | 0.905 | 2.874 | |
| AACP-MCNumber of Calibration Samples=10002026.06 | 0.905 | 1.755 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 0.903 | 1.774 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 0.902 | 1.537 | |
| RetrainNumber of Calibration Samples=3002026.06 | 0.902 | 3.258 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 0.902 | 1.563 | |
| RetrainNumber of Calibration Samples=2002026.06 | 0.901 | 3.866 | |
| AACP-ACCNumber of Calibration Samples=3002026.06 | 0.901 | 2.869 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 0.901 | 1.827 | |
| RetrainNumber of Calibration Samples=10002026.06 | 0.901 | 2.43 | |
| AACP-MCNumber of Calibration Samples=20002026.06 | 0.901 | 1.738 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 0.901 | 1.738 | |
| AACP-MCNumber of Calibration Samples=50002026.06 | 0.901 | 1.703 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 0.901 | 1.705 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 0.9 | 1.497 | |
| RetrainNumber of Calibration Samples=5002026.06 | 0.9 | 2.649 | |
| ACP-MCNumber of Calibration Samples=3002026.06 | 0.899 | 1.834 | |
| AdaTSNumber of Calibration Samples=3002026.06 | 0.899 | 1.575 | |
| RetrainNumber of Calibration Samples=20002026.06 | 0.899 | 2.099 | |
| StandardNumber of Calibration Samples=20002026.06 | 0.899 | 1.199 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 0.899 | 1.418 | |
| RetrainNumber of Calibration Samples=50002026.06 | 0.899 | 1.655 | |
| StandardNumber of Calibration Samples=50002026.06 | 0.899 | 1.194 | |
| StandardNumber of Calibration Samples=2002026.06 | 0.898 | 1.249 | |
| AACP-MCNumber of Calibration Samples=3002026.06 | 0.898 | 1.862 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 0.898 | 1.433 | |
| StandardNumber of Calibration Samples=10002026.06 | 0.898 | 1.193 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 0.898 | 1.442 | |
| AACP-ACCNumber of Calibration Samples=2002026.06 | 0.897 | 3.756 | |
| StandardNumber of Calibration Samples=3002026.06 | 0.895 | 1.213 | |
| StandardNumber of Calibration Samples=5002026.06 | 0.894 | 1.215 |