Multilingual Language Understanding on MMMLU (Massive Multilingual Language Understanding)
79.5AccuracyQwen-3-30B-A3B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen-3-30B-A3BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 79.5 | |
| Qwen-3-32BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 79 | |
| Llama-3.3-70BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 78.4 | |
| Mistral-3.2-24BOpenness=Open-weights, Region=European, Tuning=Instruction-tuned2026.02 | 74.8 | |
| Qwen-3-14BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 74.7 | |
| Gemma-3-27BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 74.6 | |
| Gemma-3-12BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 69 | |
| OLMo-3.1-32BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.02 | 66.5 | |
| EuroLLM-22B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 64.1 | |
| Apertus-70BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 60.3 | |
| EuroLLM-9B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 60.2 | |
| EuroLLM-22B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 59.7 | |
| Qwen2.5-VL-3B# Total Params=3.75B, Evaluation Toolkit=internal language eval suite2025.11 | 54.23 | |
| EuroLLM-9B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 53.8 | |
| LFM2-VL-3B# Total Params=3.0B, Evaluation Toolkit=internal language eval suite2025.11 | 53.37 | |
| Apertus-8BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 53 | |
| Llama-3.1-8BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 52.9 | |
| InternVL3.5-2B# Total Params=2.3B, Evaluation Toolkit=internal language eval suite2025.11 | 49.32 | |
| OLMo-3-7BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.02 | 48.3 | |
| Qwen3-VL-2B# Total Params=2.1B, Evaluation Toolkit=internal language eval suite2025.11 | 42.2 | |
| SmolVLM2-2.2B# Total Params=2.2B, Evaluation Toolkit=internal language eval suite2025.11 | 32.16 |