Multi-task Question Answering on ARC, OBQA, and Riddle Combined (test)
62.69AccuracyREALM
Evaluation results
| Method | Links | |
|---|---|---|
| REALMModel Size=Large (64.76%), Distribution=Dist. 2, Noise Type=Systematic2026.04 | 62.69 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 62.67 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 2, Noise Type=Uniform2026.04 | 62.41 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 3, Noise Type=Uniform2026.04 | 62.32 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 61.66 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 2, Noise Type=Uniform2026.04 | 61.49 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 1, Noise Type=Uniform2026.04 | 61.38 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 3, Noise Type=Systematic2026.04 | 61.08 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 2, Noise Type=Systematic2026.04 | 60.92 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 1, Noise Type=Asymmetric2026.04 | 60.44 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 60.44 | |
| REALMModel Size=Large (64.76%), Distribution=Dist. 1, Noise Type=Systematic2026.04 | 60.14 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 3, Noise Type=Uniform2026.04 | 59.1 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 3, Noise Type=Systematic2026.04 | 58.46 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 57.77 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 1, Noise Type=Uniform2026.04 | 54.9 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 1, Noise Type=Systematic2026.04 | 53.91 | |
| NOISYModel Size=Large (64.76%), Distribution=Dist. 1, Noise Type=Asymmetric2026.04 | 48.69 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 44.18 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 2, Noise Type=Uniform2026.04 | 44.07 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 2, Noise Type=Systematic2026.04 | 43.59 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 3, Noise Type=Systematic2026.04 | 42.83 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 2, Noise Type=Systematic2026.04 | 42.67 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 2, Noise Type=Uniform2026.04 | 42.64 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 42.62 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 3, Noise Type=Uniform2026.04 | 42.51 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 42.48 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 1, Noise Type=Uniform2026.04 | 42.41 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 1, Noise Type=Systematic2026.04 | 42.05 | |
| REALMModel Size=Base (46.05%), Distribution=Dist. 1, Noise Type=Asymmetric2026.04 | 41.93 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 41.08 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 3, Noise Type=Systematic2026.04 | 40.94 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 3, Noise Type=Uniform2026.04 | 40.3 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 1, Noise Type=Uniform2026.04 | 38.44 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 1, Noise Type=Asymmetric2026.04 | 38.25 | |
| NOISYModel Size=Base (46.05%), Distribution=Dist. 1, Noise Type=Systematic2026.04 | 37.75 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 28.71 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 2, Noise Type=Asymmetric2026.04 | 28.62 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 2, Noise Type=Uniform2026.04 | 28.57 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 2, Noise Type=Uniform2026.04 | 28.39 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 2, Noise Type=Systematic2026.04 | 28.39 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 3, Noise Type=Uniform2026.04 | 28.16 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 3, Noise Type=Systematic2026.04 | 28.16 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 27.98 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 1, Noise Type=Systematic2026.04 | 27.95 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 2, Noise Type=Systematic2026.04 | 27.93 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 1, Noise Type=Uniform2026.04 | 27.89 | |
| REALMModel Size=Small (28.57%), Distribution=Dist. 1, Noise Type=Asymmetric2026.04 | 27.49 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 3, Noise Type=Asymmetric2026.04 | 27.01 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 3, Noise Type=Uniform2026.04 | 26.74 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 3, Noise Type=Systematic2026.04 | 26.3 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 1, Noise Type=Asymmetric2026.04 | 25.68 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 1, Noise Type=Uniform2026.04 | 25.2 | |
| NOISYModel Size=Small (28.57%), Distribution=Dist. 1, Noise Type=Systematic2026.04 | 24.92 |