Multi-task Language Understanding on Global MMLU-Lite Māori
54.64AccuracyGemma-3 (27B-IT)
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemma-3 (27B-IT)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models, Parameters=27B2026.01 | 54.64 | |
| BYOL-mri (12B-M)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models, Parameters=12B2026.01 | 52.48 | |
| BYOL-mri (4B-M)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models, Parameters=4B2026.01 | 47.64 | |
| Qwen-3 (14B)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models, Parameters=14B2026.01 | 44.68 | |
| Gemma-3 (12B-IT)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models, Parameters=12B2026.01 | 43.52 | |
| Gemma-3 (4B-IT)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models, Parameters=4B2026.01 | 35.1 | |
| GPT-OSS (20B)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models, Parameters=20B2026.01 | 34.61 | |
| Qwen-3 (8B)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models, Parameters=8B2026.01 | 33.27 | |
| Apertus (8B-Inst-2509)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models, Parameters=8B2026.01 | 31.05 | |
| Llama-3.1 (Instruct-8B)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models, Parameters=8B2026.01 | 30.44 |