Arabic Language Evaluation on Almieyar
80.7Phonology ScoreFanar-27B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Fanar-27BSize=27B, Evaluation Protocol=0-shot2026.03 | 80.7 | 67.7 | 82.7 | 80.7 | 85.5 | |
| Fanar-1-9b-instructSize=9B, Evaluation Protocol=0-shot2026.03 | 65 | 63.9 | 66.3 | 74.7 | 78.7 | |
| Jais-2-70B-ChatSize=70B, Evaluation Protocol=0-shot2026.03 | 60.7 | 65.8 | 67.3 | 78 | 84.6 | |
| Qwen3-32BSize=32B, Evaluation Protocol=0-shot2026.03 | 60.7 | 67.1 | 75.5 | 64 | 68.6 | |
| KarnakSize=40B, Evaluation Protocol=0-shot2026.03 | 60 | 58.9 | 66.3 | 70.7 | 78 | |
| Gemma-3-27B-itSize=27B, Evaluation Protocol=0-shot2026.03 | 60 | 63.3 | 72.4 | 74.7 | 82 | |
| Allam-7B-Instruct-preview-v2Size=7B, Evaluation Protocol=0-shot2026.03 | 58.6 | 48.1 | 71.4 | 76.7 | 70.3 | |
| Llama-3.3-70b-InstructSize=70B, Evaluation Protocol=0-shot2026.03 | 57.1 | 77.2 | 66.3 | 68 | 73 | |
| AceGPT-v2-70B-ChatSize=70B, Evaluation Protocol=0-shot2026.03 | 47.1 | 61.4 | 40.8 | 60.7 | 61.8 | |
| AceGPT-v2-32B-ChatSize=32B, Evaluation Protocol=0-shot2026.03 | 45.7 | 60.8 | 45.9 | 60.7 | 63.1 |