Language Modeling on CulturaX (val)
47.4CulturaX Score (ar)X-ELM
Evaluation Results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| X-ELM#Param.=13.89B2025.06 | 47.4 | 52.1 | 6.6 | 45 | 11.5 | 8.1 | 2.9 | 21.6 | 31.3 | 25.9 | 36.4 | 2.1 | 2.5 | 2.8 | 13.2 | 8.7 | 174.2 | 29.3 | |
| DMoE (9 Groups)#Param.=5.19B2025.06 | 47.9 | 51.7 | 6.6 | 45.9 | 11.8 | 8.1 | 3 | 22.4 | 25.6 | 21.6 | 37.4 | 2.1 | 2.5 | 2.7 | 13.5 | 7.2 | 103.4 | 24.7 | |
| Qwen-1.8B + Pre-train#Param.=1.54B2025.06 | 48.6 | 52.7 | 6.6 | 47.1 | 11.9 | 8.3 | 3 | 22.5 | 26.9 | 22.6 | 38 | 2.1 | 2.5 | 2.8 | 13.6 | 7.4 | 121 | 26.1 | |
| Qwen-1.8B#Param.=1.54B2025.06 | 59.4 | 70.8 | 7 | 64.4 | 13.2 | 12.2 | 3 | 25.1 | 136.2 | 86.4 | 45 | 2.2 | 2.8 | 3.9 | 15 | 29.6 | 4,809.7 | 312.9 | |
| DMoE (9 Groups)#Param.=1.75B2025.06 | 68.8 | 81.1 | 8 | 67.9 | 15.3 | 9.8 | 3.4 | 31.6 | 33.6 | 27.8 | 51.8 | 2.3 | 2.7 | 3 | 17.7 | 8.6 | 173 | 35.8 | |
| X-ELM#Param.=4.44B2025.06 | 74.3 | 87.9 | 8.2 | 72.9 | 15.3 | 10.7 | 3.4 | 30.3 | 51.9 | 40.2 | 55.5 | 2.3 | 2.8 | 3.2 | 18.2 | 11.9 | 404.1 | 52.8 | |
| Qwen2.5-0.5B + Pre-train#Param.=494M2025.06 | 75.7 | 89.2 | 8.3 | 75.3 | 15.9 | 10.9 | 3.4 | 32.4 | 40.6 | 32.8 | 56.9 | 2.4 | 2.8 | 3.2 | 18.7 | 9 | 192.5 | 39.9 | |
| Qwen2.5-0.5B#Param.=494M2025.06 | 110.8 | 147.3 | 9.5 | 133.8 | 19.8 | 20.2 | 3.7 | 36.8 | 336.2 | 191.4 | 85.2 | 2.5 | 3.4 | 5.4 | 23.3 | 44.7 | 210,000 | 1,262 |