Multiple-choice Language Understanding on MMLU (test)
74.36Pooled AccuracyQwen3 8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3 8Bshot=5-shot2026.06 | 74.36 | 76.44 | 12.28 | 45 | |
| Gemma 2 9Bshot=5-shot2026.06 | 71.42 | 72.54 | 15.52 | 39 | |
| Qwen3 1.7Bshot=5-shot2026.06 | 58.25 | 60.27 | 11.67 | 33.6 | |
| Gemma 3 4Bshot=5-shot2026.06 | 58.23 | 59.54 | 16.89 | 26.5 | |
| Gemma 2 2Bshot=5-shot2026.06 | 53.23 | 54.61 | 15.01 | 22.1 | |
| Qwen3 0.6Bshot=5-shot2026.06 | 45.09 | 46.53 | 9.72 | 24.6 |