Reading Comprehension on RACE Middle School
95.4Accuracy (RACE MS)Human Ceiling
Evaluation results
| Method | Links | |
|---|---|---|
| Human CeilingType=Human baseline2021.12 | 95.4 | |
| ALBERT (ensemble)Evaluation Protocol=supervised2021.12 | 93.6 | |
| Amazon TurkType=Human baseline2021.12 | 85.1 | |
| Engram-40BShots=5-shot2026.01 | 83.3 | |
| Engram-27BShots=5-shot2026.01 | 82.8 | |
| MoE-27BShots=5-shot2026.01 | 80.9 | |
| GopherParameters=280B, Evaluation Protocol=few-shot2021.12 | 75.1 | |
| KALEBackbone=Llama3 8B2026.01 | 74.12 | |
| Dense-4BShots=5-shot2026.01 | 72.4 | |
| COT-DistillBackbone=Llama3 8B2026.01 | 69.57 | |
| PDBackbone=Llama3 8B2026.01 | 68.91 | |
| Distill-SBSBackbone=Llama3 8B2026.01 | 63.16 | |
| GPT-3Parameters=175B, Evaluation Protocol=few-shot2021.12 | 58.1 | |
| GopherParameters=7.1B, Evaluation Protocol=few-shot2021.12 | 31.8 | |
| GopherParameters=417M, Evaluation Protocol=few-shot2021.12 | 26.2 | |
| GopherParameters=1.4B, Evaluation Protocol=few-shot2021.12 | 25 |