Dialectal Arabic Evaluation on AraDiCE
68.12PIQA Acc (Egy)Fanar-27B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Fanar-27BSize=27B, Shot count=0-shot2026.03 | 68.12 | 63.66 | 66 | 68.14 | |
| AceGPT-v2-70B-ChatSize=70B, Shot count=0-shot2026.03 | 67.36 | 61.86 | 67.95 | 68.41 | |
| Jais-2-70B-ChatSize=70B, Shot count=0-shot2026.03 | 66.16 | 62.79 | 72.29 | 74.14 | |
| Gemma-3-27B-itSize=27B, Shot count=0-shot2026.03 | 65.45 | 60.34 | 64.3 | 65.64 | |
| Allam-7B-Instruct-preview-v2Size=7B, Shot count=0-shot2026.03 | 63.76 | 59.85 | 64.28 | 66.45 | |
| Fanar-1-9b-instructSize=9B, Shot count=0-shot2026.03 | 63.44 | 59.85 | 59.34 | 60.28 | |
| Llama-3.3-70b-InstructSize=70B, Shot count=0-shot2026.03 | 62.89 | 58.76 | 65.74 | 66.56 | |
| AceGPT-v2-32B-ChatSize=32B, Shot count=0-shot2026.03 | 62.62 | 60.88 | 60.97 | 62.48 | |
| Qwen3-32BSize=32B, Shot count=0-shot2026.03 | 61.7 | 56.31 | 61.6 | 62.31 | |
| KarnakSize=40B, Shot count=0-shot2026.03 | 61.64 | 59.03 | 72.4 | 75.2 |