Multiple Choice Question Answering on OBQA (test)
71.6AccuracyOracle
Evaluation Results
| Method | Links | |
|---|---|---|
| OracleModel Size=Large, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 71.6 | |
| REALMModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 69.68 | |
| REALMModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 69.6 | |
| REALMModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 69.04 | |
| NOISYModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 61.88 | |
| NOISYModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 61.36 | |
| OracleModel Size=Base, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 59.6 | |
| REALMModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 57.88 | |
| NOISYModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 57.84 | |
| REALMModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 57.52 | |
| REALMModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 57.48 | |
| NOISYModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 49.84 | |
| NOISYModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 49.12 | |
| NOISYModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 49 | |
| OracleModel Size=Small, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 44.2 | |
| REALMModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 40.8 | |
| REALMModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 39.44 | |
| REALMModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 37.84 | |
| NOISYModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 31.64 | |
| NOISYModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 30.28 | |
| NOISYModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 27.44 |