Multiple-choice Question Answering on MMLU (val)
6,402Answered CountSelf-Abstained
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Self-AbstainedAbstention strategy=natural (fifth option 'I don’t know')2026.04 | 6,402 | 619 | 91.2 | 63.67 | 1.8 | |
| Selective abstention (3 confounded features)P(correct) threshold=0.302026.04 | 6,293 | 728 | 89.6 | 65.98 | 4.1 | |
| Selective abstention (3 confounded features)P(correct) threshold=0.402026.04 | 5,389 | 1,632 | 76.8 | 70.96 | 9.1 | |
| Selective abstention (3 confounded features)P(correct) threshold=0.502026.04 | 4,522 | 2,499 | 64.4 | 75.9 | 14 | |
| Selective abstention (3 confounded features)P(correct) threshold=0.602026.04 | 3,709 | 3,312 | 52.8 | 81.29 | 19.4 | |
| Selective abstention (3 confounded features)P(correct) threshold=0.702026.04 | 3,003 | 4,018 | 42.8 | 86.55 | 24.6 |