Multilingual Language Understanding and Reasoning on (MMLU, TruthfulQA, Arc, IF, LogiQA)
53.4Average ScoreApertus-8B-Instruct-2509
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Apertus-8B-Instruct-2509Type=Instruction-tuned, Size=8B2026.05 | 53.4 | 55.3 | 52.4 | 36.8 | 68.9 | 29 | |
| Qwen3-4BType=Instruction-tuned, Size=4B2026.05 | 52.1 | 58.1 | 49.7 | 27.4 | 73.3 | 50 | |
| gemma-3-4b-itType=Instruction-tuned, Size=4B2026.05 | 49.7 | 54.7 | 49.2 | 31.6 | 63.5 | 41.1 | |
| EuroLLM-9B-InstructType=Instruction-tuned, Size=9B2026.05 | 48 | 52 | 46.5 | 32.2 | 61.3 | 34.5 | |
| SmolLM3-3BType=Instruction-tuned, Size=3B2026.05 | 47.9 | 50.7 | 50 | 27 | 63.7 | 36.5 | |
| Apertus-v1.1-4B-InstructType=Instruction-tuned, Size=4B2026.05 | 47.3 | 50.4 | 50.6 | 33.2 | 55 | 29.6 | |
| Qwen3-1.7BType=Instruction-tuned, Size=1.7B2026.05 | 45.7 | 47.7 | 49 | 25.1 | 61.1 | 41.4 | |
| gemma-3-1b-itType=Instruction-tuned, Size=1B2026.05 | 40.6 | 40.9 | 45.7 | 25 | 50.9 | 37.9 | |
| Qwen3-0.6BType=Instruction-tuned, Size=0.6B2026.05 | 40.1 | 37.7 | 46.4 | 22.2 | 54.1 | 35.3 | |
| Apertus-v1.1-1.5B-InstructType=Instruction-tuned, Size=1.5B2026.05 | 38.2 | 37.7 | 45.1 | 26.6 | 43.4 | 27.6 | |
| SmolLM2-1.7B-InstructType=Instruction-tuned, Size=1.7B2026.05 | 34.8 | 36.5 | 45.2 | 21.3 | 36.4 | 24.6 | |
| Apertus-v1.1-0.5B InstructType=Instruction-tuned, Size=0.5B2026.05 | 31.8 | 25.8 | 46.1 | 22.5 | 32.8 | 27.9 | |
| EuroLLM-1.7B-InstructType=Instruction-tuned, Size=1.7B2026.05 | 29.1 | 26 | 43.3 | 25 | 22.2 | 26.9 | |
| gemma-3-270m-itType=Instruction-tuned, Size=270m2026.05 | 28.9 | 24.2 | 46.5 | 21.5 | 23.6 | 20.5 |