Science Question Answering on ARC n=2390 (test)
100Performance Score (Act)Adaptive Local k-NN Certification (kNN no bias)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Adaptive Local k-NN Certification (kNN no bias)Operating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 100 | 95.9 | 4.1 | 0.19 | 1 | |
| Selective Pred.Operating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 100 | 95.9 | 4.1 | 0.19 | 1 | |
| Calibrated LearnedOperating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 100 | 95.9 | 4.1 | 0.19 | 1 | |
| Isotonic Conf.Operating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 100 | 96 | 4 | 0.18 | 1 | |
| Learned StopperOperating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 100 | 95.7 | 4.3 | 0.2 | 1 | |
| CRCOperating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 99.9 | 96 | 4 | 0.18 | 1 | |
| ConsensusOperating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 99 | 95.9 | 4.1 | 0.19 | 1.1 | |
| OracleOperating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 96.4 | 100 | 0 | 0 | 1.1 | |
| Confidence Thr.Operating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 83 | 95.9 | 4.1 | 0.19 | 1.9 | |
| Adaptive Local k-NN CertificationOperating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 75.5 | 97.1 | 2.3 | 0.11 | 2.3 | |
| Adaptive Local k-NN Certification (Final-round)Operating Budget (T)=4, Scaling parameter (λ*)=4.4, Target budget (β)=.22, Training error (eT)=.052026.06 | 62.3 | 96.9 | 1.9 | 0.09 | 4 |