Selective Prediction on CLINC-150 v1 (test)
95.5Performance (α=0.10)WSR Betting + LTT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| WSR Betting + LTTn_cal=11,250, delta (δ)=0.10, confidence scores=simulated2026.03 | 95.5 | 42.7 | 60.2 | 85.7 | 100 | 100 | |
| Clopper-Pearson + LTTn_cal=11,250, delta (δ)=0.10, confidence scores=simulated2026.03 | 95.2 | 44.8 | 61.9 | 82.2 | 100 | 100 | |
| LTT + Hoeffdingn_cal=11,250, delta (δ)=0.10, confidence scores=simulated2026.03 | 94.3 | — | 48.9 | 79 | 100 | 100 | |
| PAC-Bayes-λn_cal=11,250, delta (δ)=0.10, confidence scores=simulated2026.03 | 93.9 | 38.8 | 56.5 | 79 | 99.4 | 100 | |
| Hoeffding + Unionn_cal=11,250, delta (δ)=0.10, confidence scores=simulated2026.03 | 93.2 | — | 19.1 | 74.1 | 99.4 | 100 | |
| Wassersteinepsilon (ε)=0.01, n_cal=11,250, delta (δ)=0.10, confidence scores=simulated2026.03 | 91.3 | — | — | 66.8 | 98.3 | 100 | |
| CVaRbeta (β)=0.20, n_cal=11,250, delta (δ)=0.10, confidence scores=simulated2026.03 | 19.1 | — | — | — | 53 | 66.8 |