Multilingual Multiple-Choice Reasoning on Global MMLU 42 languages 1.0 (test)
54.8Average AccuracyQwen3.5-4B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen3.5-4Bevaluation harness=lm-eval, parameters=4B2026.03 | 54.8 | |
| Qwen3-4Bevaluation harness=lm-eval, parameters=4B2026.03 | 49.3 | |
| Ministral-3-3Bevaluation harness=lm-eval, parameters=3B2026.03 | 46.8 | |
| Gemma3-4Bevaluation harness=lm-eval, parameters=4B2026.03 | 45.3 | |
| Tiny Aya Globalevaluation harness=lm-eval, decoding=default2026.03 | 44.9 | |
| SmolLM3-3Bevaluation harness=lm-eval, parameters=3B2026.03 | 37.2 |