Knowledge Evaluation on ArabicMMLU
81.23AccuracyKarnak
Evaluation results
| Method | Links | |
|---|---|---|
| KarnakSize=40B, Protocol=3-shot2026.03 | 81.23 | |
| Jais-2-70B-ChatSize=70B, Protocol=3-shot2026.03 | 79.02 | |
| Fanar-27BSize=27B, Protocol=3-shot2026.03 | 74.67 | |
| AceGPT-v2-70B-ChatSize=70B, Protocol=3-shot2026.03 | 73.87 | |
| Llama-3.3-70b-InstructSize=70B, Protocol=3-shot2026.03 | 73.66 | |
| Qwen3-32BSize=32B, Protocol=3-shot2026.03 | 73.08 | |
| Gemma-3-27B-itSize=27B, Protocol=3-shot2026.03 | 72.21 | |
| Allam-7B-Instruct-preview-v2Size=7B, Protocol=3-shot2026.03 | 70.01 | |
| AceGPT-v2-32B-ChatSize=32B, Protocol=3-shot2026.03 | 69.55 | |
| Fanar-1-9b-instructSize=9B, Protocol=3-shot2026.03 | 67.35 |