Multilingual Multiple-Choice Reasoning on INCLUDE 44 languages 1.0 (test)
56.9Average AccuracyQwen3.5-4B
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen3.5-4Bevaluation harness=lm-eval, parameters=4B2026.03 | 56.9 | |
| Ministral-3-3Bevaluation harness=lm-eval, parameters=3B2026.03 | 52.6 | |
| Qwen3-4Bevaluation harness=lm-eval, parameters=4B2026.03 | 52.2 | |
| Gemma3-4Bevaluation harness=lm-eval, parameters=4B2026.03 | 48.9 | |
| Tiny Aya Globalevaluation harness=lm-eval, decoding=default2026.03 | 45.1 | |
| SmolLM3-3Bevaluation harness=lm-eval, parameters=3B2026.03 | 39.4 |