Reasoning on MMLU-ProX Lite
83Target AccuracyGEMINI-3-FLASH
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GEMINI-3-FLASHConfiguration=LOW2026.06 | 83 | -0.8 | 98.3 | |
| GEMINI-3-FLASHConfiguration=Default2026.06 | 82.1 | -1.4 | 98.8 | |
| GEMINI-3-FLASHConfiguration=MINIMAL2026.06 | 77.2 | 3.2 | 93.5 | |
| CLAUDEModel Identity=OPUS-4.72026.06 | 76.7 | 5.8 | 89.3 | |
| GEMINI-2.5-FLASHConfiguration=Default2026.06 | 73.4 | -1.9 | 97.5 | |
| GPT-OSSParameters=120B2026.06 | 72.3 | 6.6 | 90.6 | |
| GPT-OSSParameters=20B2026.06 | 66.3 | 7.7 | 88.9 | |
| GEMINI-2.5-FLASHThinking Mode=OFF2026.06 | 61.6 | 5.9 | 88.9 | |
| CLAUDEModel Identity=SONNET-4.62026.06 | 59.5 | 7.4 | 85.4 | |
| GEMMA-4Configuration=31B-IT2026.06 | 59.4 | 4.8 | 84.7 | |
| GEMMA-4Configuration=26B-A4B-IT2026.06 | 48.7 | 4.8 | 78.7 | |
| CLAUDEModel Identity=HAIKU-4.52026.06 | 44.6 | 0.6 | 76.5 | |
| QWEN-3Parameters=30B-A3B2026.06 | 43.4 | 10.5 | 74.8 | |
| GEMMA-3Parameters=27B2026.06 | 39.4 | 6.4 | 77.2 | |
| GEMMA-4Configuration=E4B-IT2026.06 | 35.8 | 7.1 | 74.8 | |
| GEMMA-3Parameters=12B2026.06 | 34.9 | 7 | 71.6 | |
| QWEN-3Parameters=4B2026.06 | 32.7 | 8.1 | 69.3 | |
| GEMMA-4Configuration=E2B-IT2026.06 | 26.1 | 5.7 | 59.3 | |
| GEMMA-3Parameters=4B2026.06 | 24.6 | 3.6 | 67.5 | |
| GEMMA-3Parameters=1B2026.06 | 14.2 | 2.7 | 58.4 |