Massive Multitask Language Understanding on MMLU (Language-Specific Accuracy)
41.2MMLU Accuracy (ar)RA-MoE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RA-MoEModel=Qwen1.52026.05 | 41.2 | 29.4 | 53.8 | 51.6 | 43.8 | 52.1 | |
| RISEModel=Qwen1.52026.05 | 40.3 | 28.6 | 53.2 | 51 | 43.1 | 52.3 | |
| SFTModel=Qwen1.52026.05 | 40 | 28.3 | 52.8 | 50.6 | 42.7 | 51.2 | |
| RSModel=Qwen1.52026.05 | 36.4 | 24.9 | 46 | 45.8 | 38.3 | 45.1 | |
| 0-shotModel=Qwen1.52026.05 | 35.8 | 24.3 | 46.7 | 45.2 | 37.9 | 45.4 | |
| RISEModel=DSV22026.05 | 32.9 | 29.3 | 50.1 | 45.2 | 42.5 | 48.3 | |
| RA-MoEModel=DSV22026.05 | 32.8 | 29.7 | 50.6 | 45 | 43.2 | 48.8 | |
| RSModel=DSV22026.05 | 32.5 | 28.9 | 49.3 | 44.4 | 41.1 | 47.9 | |
| SFTModel=DSV22026.05 | 32.3 | 29.2 | 50 | 45 | 42.2 | 48.2 | |
| 0-shotModel=DSV22026.05 | 32.1 | 28.8 | 49.9 | 44.7 | 40.8 | 47.4 | |
| RISEModel=OLMoE2026.05 | 29.1 | 25.8 | 32.9 | 38.2 | 29.9 | 38.1 | |
| RA-MoEModel=OLMoE2026.05 | 28.9 | 26 | 32.7 | 38.6 | 30.3 | 38.6 | |
| SFTModel=OLMoE2026.05 | 28.4 | 25.6 | 32.1 | 38 | 29.7 | 37.8 | |
| 0-shotModel=OLMoE2026.05 | 28.3 | 25.5 | 31.7 | 37.6 | 29.6 | 36.9 | |
| RSModel=OLMoE2026.05 | 27.5 | 25 | 31.3 | 37.1 | 29.5 | 37.3 |