Multiple Choice Question Answering on Riddle (test)
67.84AccuracyOracle
Evaluation results
| Method | Links | |
|---|---|---|
| OracleModel Size=Large, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 67.84 | |
| REALMModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 60.55 | |
| REALMModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 60.43 | |
| REALMModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 59.69 | |
| OracleModel Size=Base, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 54.12 | |
| NOISYModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 50.75 | |
| NOISYModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 50.35 | |
| NOISYModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 50.08 | |
| REALMModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 46.86 | |
| REALMModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 46.04 | |
| REALMModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 45.76 | |
| OracleModel Size=Small, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 39.02 | |
| NOISYModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 37.14 | |
| NOISYModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 36.2 | |
| NOISYModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 35.37 | |
| REALMModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 31.88 | |
| REALMModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 31.49 | |
| REALMModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 31.45 | |
| NOISYModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 23.8 | |
| NOISYModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 23.53 | |
| NOISYModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 21.96 |