Multiple-choice Question Answering on PIQA (test)
77.8Accuracy (PIQA)Oracle
Evaluation results
| Method | Links | |
|---|---|---|
| OracleModel Size=Large, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 77.8 | |
| REALMModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 77.67 | |
| REALMModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 77.06 | |
| REALMModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 76.45 | |
| NOISYModel Size=Large, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 75.6 | |
| OracleModel Size=Base, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 67.68 | |
| REALMModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 66.92 | |
| REALMModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 66.22 | |
| REALMModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 65.7 | |
| NOISYModel Size=Base, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 64.87 | |
| AR Transformer2026.02 | 63.71 | |
| MDLM2026.02 | 57.89 | |
| REALMModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 54.41 | |
| OracleModel Size=Small, Noise Type=None, Expertise Distribution=Dist. ①2026.04 | 52.88 | |
| REALMModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 52.14 | |
| Duo++k=22026.02 | 52.12 | |
| Duo++k=52026.02 | 51.25 | |
| Duo2026.02 | 51.14 | |
| NOISYModel Size=Small, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 51.08 | |
| REALMModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 51.06 | |
| NOISYModel Size=Small, Noise Type=Systematic, Expertise Distribution=Dist. ①2026.04 | 50.84 | |
| Duo++k=32026.02 | 50.65 | |
| NOISYModel Size=Small, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 50.16 | |
| NOISYModel Size=Base, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 46.31 | |
| NOISYModel Size=Base, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 46.31 | |
| NOISYModel Size=Large, Noise Type=Asymmetric, Expertise Distribution=Dist. ①2026.04 | 27.62 | |
| NOISYModel Size=Large, Noise Type=Uniform, Expertise Distribution=Dist. ①2026.04 | 26.27 |