Language Understanding on MMLU (AUROC, Tier, and r)
83.7MMLU AUROCClaude Sonnet 4.6
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Claude Sonnet 4.62026.04 | 83.7 | — | 0.25 | |
| Claude Opus 4.62026.04 | 83.6 | — | 0.217 | |
| Gemini 2.5 Pro2026.04 | 80.3 | — | 0.558 | |
| Gemini 3.1 Pro2026.04 | 78.6 | — | 0.294 | |
| DeepSeek-R12026.04 | 76.8 | — | 0.204 | |
| Claude Haiku 4.52026.04 | 76 | — | 0.283 | |
| Gemini 2.5 Flash2026.04 | 72.4 | — | 0.246 | |
| Qwen 80B Think2026.04 | 71.8 | — | 0.265 | |
| GPT-5.42026.04 | 71.5 | — | 0.19 | |
| GPT-5.4 mini2026.04 | 69.9 | — | 0.235 | |
| Gemini 3 Flash2026.04 | 69.5 | — | 0.201 | |
| GLM-52026.04 | 67.9 | — | 0.135 | |
| Qwen 235B2026.04 | 64.7 | — | 0.221 | |
| Qwen 80B Instruct2026.04 | 61.3 | — | 0.141 | |
| Gemma 3 27B2026.04 | 58.8 | — | 0.147 | |
| GPT-5.4 nano2026.04 | 55.4 | — | 0.066 | |
| Gemma 3 12B2026.04 | 53.3 | — | 0.106 | |
| Gemma 3 1B2026.04 | 50 | — | 0 |