Question Answering on ARC-Hard chat (Māori)
59.22AccuracyBYOL-mri (12B-M)
Evaluation results
| Method | Links | |
|---|---|---|
| BYOL-mri (12B-M)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models2026.01 | 59.22 | |
| BYOL-mri (4B-M)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models2026.01 | 51.11 | |
| Gemma-3 (27B-IT)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models2026.01 | 49.66 | |
| Gemma-3 (12B-IT)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models2026.01 | 43.09 | |
| Gemma-3 (4B-IT)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models2026.01 | 37.5 | |
| Qwen-3 (14B)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models2026.01 | 35.75 | |
| Qwen-3 (8B)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models2026.01 | 34.04 | |
| GPT-OSS (20B)Zero-shot=true, Instruction-tuned=true, Model scale category=12B+ Models2026.01 | 32.85 | |
| Llama-3.1 (Instruct-8B)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models2026.01 | 31.91 | |
| Apertus (8B-Inst-2509)Zero-shot=true, Instruction-tuned=true, Model scale category=4B-8B Models2026.01 | 27.3 |