Multiple Choice Question Answering on ARC (test)
64.81AccuracyOracle
Evaluation results
| Method | Links | |
|---|---|---|
| OracleModel Size=Large, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 64.81 | |
| REALMModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 57.68 | |
| REALMModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 57.56 | |
| REALMModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 57.48 | |
| NOISYModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 54.99 | |
| NOISYModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 53.68 | |
| NOISYModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 53.01 | |
| OracleModel Size=Base, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 47.47 | |
| REALMModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 43.14 | |
| REALMModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 42.76 | |
| REALMModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 42.56 | |
| NOISYModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 41.77 | |
| NOISYModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 40.74 | |
| NOISYModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 40.55 | |
| OracleModel Size=Small, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 29.44 | |
| REALMModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 27.35 | |
| REALMModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 27.04 | |
| NOISYModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 26.97 | |
| NOISYModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 26.68 | |
| REALMModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 26.51 | |
| NOISYModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 25.91 |