Question Answering on ISLAMICFAITHQA
48.2Accuracy (Arabic)Fanar-2-27B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Fanar-2-27BConfiguration=Base (non-retrieval)2026.01 | 48.2 | 47.9 | 48.05 | — | |
| ALLaM-7BConfiguration=Base (non-retrieval)2026.01 | 42.7 | 32.8 | 37.75 | — | |
| Fanar-1-9BConfiguration=Base (non-retrieval)2026.01 | 34.5 | 36.3 | 35.4 | — | |
| AceGPT-v2-8BConfiguration=Base (non-retrieval)2026.01 | 23.1 | 28.8 | 25.95 | — | |
| EuroLLM-9BConfiguration=Base (non-retrieval)2026.01 | 22.3 | 29.1 | 25.7 | — | |
| SILMA-9B-v1.0Configuration=Base (non-retrieval)2026.01 | 20.4 | 28.5 | 24.45 | — | |
| Qwen3-14BConfiguration=Base (non-retrieval)2026.01 | 16 | 14 | 15 | — | |
| gpt-oss-20bConfiguration=Base (non-retrieval)2026.01 | 15.9 | 27.2 | 21.55 | — | |
| Qwen3-4B-2507Configuration=Base (non-retrieval)2026.01 | 15.8 | 27.9 | 21.85 | — | |
| Mistral-7B-v0.2Configuration=Base (non-retrieval)2026.01 | 13.5 | 24.4 | 18.95 | — | |
| Llama-3.1-8BConfiguration=Base (non-retrieval)2026.01 | 13 | 25.8 | 19.4 | — | |
| SeaLLM-7B-v3Configuration=Base (non-retrieval)2026.01 | 11.6 | 23.8 | 17.7 | — | |
| Qwen2.5-3BConfiguration=Base (non-retrieval)2026.01 | 11 | 20 | 15.5 | — | |
| Qwen3-8BConfiguration=Base (non-retrieval)2026.01 | 8.8 | 8.5 | 8.65 | — | |
| Llama-2-7bConfiguration=Base (non-retrieval)2026.01 | 4.4 | 18.8 | 11.6 | — | |
| ALLaMEvaluation Protocol=LLM-as-a-judge, Parameters=7B2026.03 | — | — | — | 42.7 | |
| FanarEvaluation Protocol=LLM-as-a-judge, Parameters=2-27B2026.03 | — | — | — | 48.2 | |
| Fanar-SadiqEvaluation Protocol=LLM-as-a-judge2026.03 | — | — | — | 65.4 | |
| Gemini-3-FlashEvaluation Protocol=LLM-as-a-judge2026.03 | — | — | — | 53.4 | |
| Gemini-3-ProEvaluation Protocol=LLM-as-a-judge2026.03 | — | — | — | 56.6 | |
| GPT-4.1Evaluation Protocol=LLM-as-a-judge2026.03 | — | — | — | 41.4 | |
| GPT-5Evaluation Protocol=LLM-as-a-judge2026.03 | — | — | — | 51.2 |