Multitask Language Understanding on MMMLU Swahili 1.0 (test)
33.38AccuracyCLO
Evaluation Results
| Method | Links | |
|---|---|---|
| CLOBase Model=Llama-3-8B, Training Data=English + Swahili2025.05 | 33.38 | |
| CLOBase Model=Qwen2.5-3B, Training Data=English + Swahili2025.05 | 29.8 | |
| SFT-tgtBase Model=Llama-3-8B, Training Data=Target language only2025.05 | 29.05 | |
| SFT+DPOBase Model=Llama-3-8B, Training Data=English + Swahili2025.05 | 28.86 | |
| SFT-tgtBase Model=Mistral-7B, Training Data=Target language only2025.05 | 28.61 | |
| SFT+DPOBase Model=Mistral-7B, Training Data=English + Swahili2025.05 | 28.42 | |
| CLOBase Model=Mistral-7B, Training Data=English + Swahili2025.05 | 28.07 | |
| SFTBase Model=Llama-3-8B, Training Data=English + Swahili2025.05 | 27.59 | |
| CLOBase Model=Llama-2-13B, Training Data=English + Swahili2025.05 | 26.78 | |
| SFTBase Model=Mistral-7B, Training Data=English + Swahili2025.05 | 26.68 | |
| SFTBase Model=Qwen2.5-3B, Training Data=English + Swahili2025.05 | 26.22 | |
| CLOBase Model=Llama-2-7B, Training Data=English + Swahili2025.05 | 24.09 | |
| SFT-tgtBase Model=Llama-2-13B, Training Data=Target language only2025.05 | 22.56 | |
| SFT-tgtBase Model=Llama-2-7B, Training Data=Target language only2025.05 | 22.46 | |
| SFTBase Model=Llama-2-13B, Training Data=English + Swahili2025.05 | 21.38 | |
| SFT+DPOBase Model=Llama-2-7B, Training Data=English + Swahili2025.05 | 19.96 | |
| SFT+DPOBase Model=Llama-2-13B, Training Data=English + Swahili2025.05 | 19.83 | |
| SFTBase Model=Llama-2-7B, Training Data=English + Swahili2025.05 | 17.21 |