Question Answering on ARC n=2590 (test)
100ActivationSelective Prediction
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Selective Predictionbeta=0.15, T=42026.06 | 100 | 95.7 | 4.3 | |
| Calibrated Learnedbeta=0.15, T=42026.06 | 100 | 95.7 | 4.3 | |
| Isotonic Confidencebeta=0.15, T=42026.06 | 100 | 95.7 | 4.3 | |
| Learned Stopperbeta=0.15, T=42026.06 | 100 | 95.7 | 4.3 | |
| CRCbeta=0.15, T=42026.06 | 99.9 | 95.8 | 4.2 | |
| Consensusbeta=0.15, T=42026.06 | 99 | 95.7 | 4.3 | |
| Oraclebeta=0.15, T=42026.06 | 96.3 | 100 | 0 | |
| Confidence Thresholdbeta=0.15, T=42026.06 | 82.8 | 95.7 | 4.3 | |
| kNN no biasbeta=0.15, T=42026.06 | 32.6 | 97.6 | 0.9 | |
| Final-roundbeta=0.15, T=42026.06 | 0.2 | 9.6 | 0 | |
| Adaptive Local k-NN Certification (Ours)beta=0.15, T=42026.06 | 0 | — | 0 |