Reading Comprehension on Belebele (Xho, Zul, Tsn, Ssw, Sot, Eng, Afr)
65.9Reading Comprehension Score (Xho)Aya-101
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Aya-101Size=13B2026.03 | 65.9 | 64.9 | 63.6 | 57.6 | 61.7 | 86.1 | 81.7 | |
| XLM-V largeSize=-2026.03 | 54.4 | 54.2 | — | 47.1 | 32.7 | 77.8 | 72.3 | |
| Meta-Llama-3-70B-InstructSize=70B2026.03 | 41.3 | 42.9 | 41.4 | 42.9 | 51.3 | 93.2 | 88.9 | |
| Llama-3.1-8B-InstructSize=8B2026.03 | 35.1 | 35.3 | 32.3 | 32.3 | 33.7 | 80.7 | 66.9 | |
| MzansiLM Base 1-shotSize=0.125B2026.03 | 29.4 | 28.2 | 30.8 | 29.7 | 27.4 | 27.4 | 27.6 | |
| MzansiLM Base 3-shotSize=0.125B2026.03 | 28.8 | 28.2 | 29 | 28.8 | 27.3 | 27.6 | 27.4 | |
| MzansiLM Base 0-shotSize=0.125B2026.03 | 27.8 | 28 | 28.3 | 27.8 | 27.1 | 27.3 | 27.3 | |
| AfroLlama-V1Size=8B2026.03 | 24.8 | 28.2 | 26.8 | 22.9 | 22.6 | 25.3 | 24.7 | |
| InkubaLM-0.4BSize=0.4B2026.03 | 23.1 | 23.2 | 24.6 | — | — | 23.9 | 25.9 | |
| MzansiLM general-ftSize=0.125B2026.03 | 21.9 | 25.1 | 21.9 | 23.6 | 22 | 29.6 | 31.1 |