Selective Prediction on MASSIVE (test)
100Guaranteed Test Coverage (alpha=0.10)Clopper-Pearson + LTT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Clopper-Pearson + LTTdelta=0.10, n_cal=549, K=1002026.03 | 100 | 48.5 | 59.1 | 83.9 | 100 | 100 | |
| WSR Betting + LTTdelta=0.10, n_cal=549, K=1002026.03 | 96 | 6 | 62.7 | 83.9 | 100 | 100 | |
| LTT + Hoeffdingdelta=0.10, n_cal=549, K=1002026.03 | 94 | — | — | 51.5 | 100 | 100 | |
| LTT + Emp. Bernsteindelta=0.10, n_cal=549, K=1002026.03 | 94 | — | 6 | 67.3 | 100 | 100 | |
| PAC-Bayes-λdelta=0.10, n_cal=549, K=1002026.03 | 94 | 6 | 51.5 | 70.5 | 100 | 100 | |
| Emp. Bernstein + Uniondelta=0.10, n_cal=549, K=1002026.03 | 79.6 | — | — | 6 | 96 | 100 | |
| Hoeffding + Uniondelta=0.10, n_cal=549, K=1002026.03 | 73.8 | — | — | — | 99.8 | 100 | |
| Wasserstein (ε=0.01)delta=0.10, n_cal=549, K=100, epsilon=0.012026.03 | 62.7 | — | — | — | 96 | 100 |