Multitask Language Understanding on GlobalMMLU
73.3AccuracyMarco-Mini-Instruct
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Marco-Mini-InstructEvaluation Protocol=Acc@12026.04 | 73.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SARAAnchor=English2026.06 | 71.53 | — | 72.7 | — | — | — | — | — | 61 | 65.76 | — | 74.66 | 72.6 | 82.45 | — | |
| ShifConAnchor=English2026.06 | 71.12 | — | 72.18 | — | — | — | — | — | 60.15 | 65.68 | — | 74.32 | 72.25 | 82.12 | — | |
| FFTAnchor=English2026.06 | 70.73 | — | 71.72 | — | — | — | — | — | 59.39 | 65.6 | — | 73.97 | 71.91 | 81.8 | — | |
| Qwen3-4B-InstructEvaluation Protocol=Acc@12026.04 | 70.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SARAAnchor=Chinese2026.06 | 69.3 | — | 70.84 | — | — | — | — | — | 59.46 | 64.68 | — | 72.62 | 70.9 | — | 77.27 | |
| Gemma3-12B-InstructEvaluation Protocol=Acc@12026.04 | 69.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ShifConAnchor=Chinese2026.06 | 69.02 | — | 70.53 | — | — | — | — | — | 58.65 | 64.48 | — | 72.55 | 70.64 | — | 77.24 | |
| FFTAnchor=Chinese2026.06 | 68.78 | — | 70.23 | — | — | — | — | — | 57.88 | 64.28 | — | 72.69 | 70.38 | — | 77.2 | |
| AESAnchor=English2026.06 | 68.63 | — | 70.04 | — | — | — | — | — | 56.09 | 63.58 | — | 71.53 | 69.52 | 81.03 | — | |
| Granite4-Small-InstructEvaluation Protocol=Acc@12026.04 | 67.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AESAnchor=Chinese2026.06 | 66.74 | — | 68.57 | — | — | — | — | — | 55.09 | 62.13 | — | 69.85 | 68.24 | — | 76.54 | |
| Vanilla LMAnchor=None2026.06 | 59.28 | — | 64.86 | — | — | — | — | — | 23.66 | 57.46 | — | 68.15 | 63.2 | 79.65 | 76.97 | |
| Marco-Nano-InstructEvaluation Protocol=Acc@12026.04 | 58.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LFM2-24B-A2B-InstructEvaluation Protocol=Acc@12026.04 | 57 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ministral-3-8B-InstructEvaluation Protocol=Acc@12026.04 | 55.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LFM2-8B-A1B-InstructEvaluation Protocol=Acc@12026.04 | 49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen32025.12 | 48.65 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.52025.12 | 47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-1.7B-InstructEvaluation Protocol=Acc@12026.04 | 46.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-VL-2B-InstructEvaluation Protocol=Acc@12026.04 | 45.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Granite4-Tiny-InstructEvaluation Protocol=Acc@12026.04 | 43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SARABackbone=Phi-3.5-MoE-instruct, Semantic Anchor=English2026.06 | 42.8 | — | 28.7 | — | — | — | — | — | 45.66 | 19.09 | — | 44.14 | 38.25 | 80.98 | — | |
| ShifConBackbone=Phi-3.5-MoE-instruct, Semantic Anchor=English2026.06 | 42.22 | — | 28.35 | — | — | — | — | — | 45.54 | 17.55 | — | 43.15 | 37.82 | 80.92 | — | |
| FFTBackbone=Phi-3.5-MoE-instruct, Semantic Anchor=English2026.06 | 41.63 | — | 27.99 | — | — | — | — | — | 45.42 | 15.96 | — | 42.14 | 37.39 | 80.86 | — | |
| Gamayun2025.12 | 40.58 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AESBackbone=Phi-3.5-MoE-instruct, Semantic Anchor=English2026.06 | 39.84 | — | 26.54 | — | — | — | — | — | 43.2 | 12.45 | — | 40.55 | 36.12 | 80.15 | — | |
| Llama3.22025.12 | 39.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Ministral-3-3B-InstructEvaluation Protocol=Acc@12026.04 | 38.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma32025.12 | 36.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EuroLM2025.12 | 26.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Vanilla LMBackbone=Phi-3.5-MoE-instruct, Semantic Anchor=English2026.06 | 15.88 | — | 12.78 | — | — | — | — | — | 7.59 | 0.27 | — | 13.58 | 11.88 | 49.16 | — | |
| Granite 4 MicroVariant=Base2026.01 | — | 27.9 | 40.5 | 31.1 | 32.5 | 33.8 | 32.8 | 30.4 | 32.9 | 38.9 | 31.1 | — | — | — | — | |
| KakugoVariant=GenReas+Tran (full)2026.01 | — | 29.4 | 40.7 | 33.2 | 35.1 | 37.5 | 35.1 | 32.3 | 35.1 | 38.7 | 33.4 | — | — | — | — |