Graduate-level Science Question Answering on GPQA n=546 (test)
100Act ScoreAdaptive Local k-NN Certification (kNN no bias)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Adaptive Local k-NN Certification (kNN no bias)Operating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 100 | 60.5 | 39.5 | 51 | 1 | |
| CRCOperating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 99.9 | 63 | 36.9 | 48 | 1.2 | |
| Selective Pred.Operating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 98.6 | 68.4 | 31.2 | 41 | 1.3 | |
| Isotonic Conf.Operating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 96.7 | 70.4 | 28.6 | 37 | 1.3 | |
| Calibrated LearnedOperating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 96.6 | 66.1 | 32.9 | 43 | 1.4 | |
| ConsensusOperating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 87 | 71.3 | 28.7 | 37 | 1.8 | |
| OracleOperating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 74.5 | 100 | 0 | 0 | 1.9 | |
| Adaptive Local k-NN Certification (Final-round)Operating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 60.9 | 82.7 | 10.6 | 14 | 4 | |
| Adaptive Local k-NN CertificationOperating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 53.3 | 82.5 | 9.4 | 12 | 3.1 | |
| Confidence Thr.Operating Budget (T)=4, Scaling parameter (λ*)=2.8, Target budget (β)=.77, Training error (eT)=.282026.06 | 50.5 | 71.9 | 28.1 | 37 | 2.8 |