Conformal Prediction on 5-class Synthetic Data Example 2 covariate shift a=0.5 Marginal
92CoverageACP-ACC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACP-ACCNumber of Calibration Samples=2002026.06 | 92 | 2.341 | |
| ACP-ACCNumber of Calibration Samples=2002026.06 | 92 | 2.341 | |
| TrustscoreNumber of Calibration Samples=2002026.06 | 90.9 | 1.942 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 90.7 | 2.043 | |
| ACP-ACCNumber of Calibration Samples=5002026.06 | 90.7 | 2.043 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 90.6 | 1.929 | |
| ACP-AECNumber of Calibration Samples=2002026.06 | 90.6 | 1.929 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 90.4 | 1.927 | |
| StandardNumber of Calibration Samples=5002026.06 | 90.4 | 1.833 | |
| AdaTSNumber of Calibration Samples=5002026.06 | 90.4 | 1.927 | |
| StandardNumber of Calibration Samples=5002026.06 | 90.4 | 1.833 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 90.3 | 1.944 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 90.3 | 1.999 | |
| RetrainNumber of Calibration Samples=10002026.06 | 90.3 | 3.287 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 90.3 | 1.947 | |
| AdaTSNumber of Calibration Samples=2002026.06 | 90.3 | 1.944 | |
| TSNumber of Calibration Samples=2002026.06 | 90.3 | 1.933 | |
| TSNumber of Calibration Samples=5002026.06 | 90.3 | 1.917 | |
| TrustscoreNumber of Calibration Samples=5002026.06 | 90.3 | 1.88 | |
| ACP-ACCNumber of Calibration Samples=10002026.06 | 90.3 | 1.999 | |
| RetrainNumber of Calibration Samples=10002026.06 | 90.3 | 3.287 | |
| ACP-ACCNumber of Calibration Samples=20002026.06 | 90.3 | 1.947 | |
| ACP-MC-OracleNumber of Calibration Samples=20002026.06 | 90.3 | 1.613 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 90.2 | 2.236 | |
| StandardNumber of Calibration Samples=2002026.06 | 90.2 | 1.846 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 90.2 | 1.89 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 90.2 | 1.943 | |
| ACP-MCNumber of Calibration Samples=2002026.06 | 90.2 | 2.236 | |
| StandardNumber of Calibration Samples=2002026.06 | 90.2 | 1.846 | |
| ACP-AECNumber of Calibration Samples=5002026.06 | 90.2 | 1.89 | |
| AdaTSNumber of Calibration Samples=10002026.06 | 90.2 | 1.943 | |
| ACP-MC-OracleNumber of Calibration Samples=50002026.06 | 90.2 | 1.61 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 90.1 | 2.218 | |
| StandardNumber of Calibration Samples=10002026.06 | 90.1 | 1.852 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 90.1 | 2.132 | |
| RetrainNumber of Calibration Samples=20002026.06 | 90.1 | 3.063 | |
| StandardNumber of Calibration Samples=20002026.06 | 90.1 | 1.824 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 90.1 | 2.118 | |
| StandardNumber of Calibration Samples=50002026.06 | 90.1 | 1.876 | |
| ACP-MCNumber of Calibration Samples=5002026.06 | 90.1 | 2.218 | |
| StandardNumber of Calibration Samples=10002026.06 | 90.1 | 1.852 | |
| TSNumber of Calibration Samples=10002026.06 | 90.1 | 1.941 | |
| ACP-MCNumber of Calibration Samples=20002026.06 | 90.1 | 2.132 | |
| RetrainNumber of Calibration Samples=20002026.06 | 90.1 | 3.063 | |
| StandardNumber of Calibration Samples=20002026.06 | 90.1 | 1.824 | |
| ACP-MCNumber of Calibration Samples=50002026.06 | 90.1 | 2.118 | |
| StandardNumber of Calibration Samples=50002026.06 | 90.1 | 1.876 | |
| TSNumber of Calibration Samples=50002026.06 | 90.1 | 1.956 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 90 | 1.905 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 90 | 1.969 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 90 | 1.888 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 90 | 1.956 | |
| RetrainNumber of Calibration Samples=50002026.06 | 90 | 2.082 | |
| ACP-MC-OracleNumber of Calibration Samples=5002026.06 | 90 | 1.608 | |
| ACP-MC-OracleNumber of Calibration Samples=10002026.06 | 90 | 1.609 | |
| AdaTSNumber of Calibration Samples=20002026.06 | 90 | 1.905 | |
| TSNumber of Calibration Samples=20002026.06 | 90 | 1.902 | |
| TrustscoreNumber of Calibration Samples=20002026.06 | 90 | 1.827 | |
| ACP-ACCNumber of Calibration Samples=50002026.06 | 90 | 1.969 | |
| ACP-AECNumber of Calibration Samples=50002026.06 | 90 | 1.888 | |
| AdaTSNumber of Calibration Samples=50002026.06 | 90 | 1.956 | |
| RetrainNumber of Calibration Samples=50002026.06 | 90 | 2.082 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 89.9 | 1.887 | |
| ACP-AECNumber of Calibration Samples=10002026.06 | 89.9 | 1.887 | |
| TrustscoreNumber of Calibration Samples=50002026.06 | 89.9 | 1.881 | |
| TrustscoreNumber of Calibration Samples=10002026.06 | 89.8 | 1.853 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 89.7 | 2.17 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 89.7 | 1.878 | |
| ACP-MC-OracleNumber of Calibration Samples=2002026.06 | 89.7 | 1.602 | |
| ACP-MCNumber of Calibration Samples=10002026.06 | 89.7 | 2.17 | |
| ACP-AECNumber of Calibration Samples=20002026.06 | 89.7 | 1.878 | |
| RetrainNumber of Calibration Samples=2002026.06 | 86.7 | 3.921 | |
| RetrainNumber of Calibration Samples=2002026.06 | 86.7 | 3.921 | |
| RetrainNumber of Calibration Samples=5002026.06 | 86.5 | 3.276 | |
| RetrainNumber of Calibration Samples=5002026.06 | 86.5 | 3.276 |