Language Understanding on MMLU-Redux Generative
87.8Humanities AccuracyGPT-5 nano
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5 nanoevaluation protocol=generative, match type=exact match2026.05 | 87.8 | 89.1 | 88.6 | 92.7 | |
| gpt-oss-20bevaluation protocol=generative, match type=exact match2026.05 | 85.8 | 87.1 | 87.6 | 93.5 | |
| Qwen3-8Bevaluation protocol=generative, match type=exact match2026.05 | 84.3 | 83 | 85.6 | 80.8 | |
| Qwen3-4Bevaluation protocol=generative, match type=exact match2026.05 | 81 | 81.4 | 83.6 | 88.8 | |
| Ministral-3-8Bevaluation protocol=generative, match type=exact match2026.05 | 78.8 | 80.5 | 84 | 69.9 | |
| gemma-3-12b-itevaluation protocol=generative, match type=exact match2026.05 | 77.4 | 78 | 83.2 | 66.4 | |
| EngGPT2-16B-A3Bevaluation protocol=generative, match type=exact match2026.05 | 72.5 | 74.7 | 75.4 | 72.4 | |
| Llama-3.1-8B-Instructevaluation protocol=generative, match type=exact match2026.05 | 72.2 | 73.5 | 74.3 | 56.9 | |
| LLaMAntino-3evaluation protocol=generative, match type=exact match2026.05 | 71.9 | 73.6 | 74.3 | 56.9 | |
| Moonlight-16B-A3B-Instructevaluation protocol=generative, match type=exact match2026.05 | 71.5 | 72.3 | 75.7 | 57.6 | |
| gemma-3-4b-itevaluation protocol=generative, match type=exact match2026.05 | 64.3 | 65.2 | 69 | 50.6 | |
| Velvet-14Bevaluation protocol=generative, match type=exact match2026.05 | 60.4 | 59.7 | 62.9 | 46.7 | |
| Llama-3.2-3B-Instructevaluation protocol=generative, match type=exact match2026.05 | 59.5 | 61.9 | 60 | 44.7 | |
| FastwebMIIA-7Bevaluation protocol=generative, match type=exact match2026.05 | 46.4 | 45.4 | 46.9 | 38.6 | |
| deepseek-moe-16b-chatevaluation protocol=generative, match type=exact match2026.05 | 40.2 | 39.7 | 43.9 | 30.8 | |
| Minerva-7B-instruct-v1.0evaluation protocol=generative, match type=exact match2026.05 | 30.6 | 31.2 | 30.3 | 26 |