List your company
ResearchFormerly WizWand

Multitask Language Understanding on MMMLU Swahili 1.0 (test)

33.38AccuracyCLO
16.563220.929125.29529.6609May 20, 2025
Updated 8mo ago

Evaluation results

MethodLinks
CLOBase Model=Llama-3-8B, Training Data=English + Swahili2025.0533.38
CLOBase Model=Qwen2.5-3B, Training Data=English + Swahili2025.0529.8
SFT-tgtBase Model=Llama-3-8B, Training Data=Target language only2025.0529.05
SFT+DPOBase Model=Llama-3-8B, Training Data=English + Swahili2025.0528.86
SFT-tgtBase Model=Mistral-7B, Training Data=Target language only2025.0528.61
SFT+DPOBase Model=Mistral-7B, Training Data=English + Swahili2025.0528.42
CLOBase Model=Mistral-7B, Training Data=English + Swahili2025.0528.07
SFTBase Model=Llama-3-8B, Training Data=English + Swahili2025.0527.59
CLOBase Model=Llama-2-13B, Training Data=English + Swahili2025.0526.78
SFTBase Model=Mistral-7B, Training Data=English + Swahili2025.0526.68
SFTBase Model=Qwen2.5-3B, Training Data=English + Swahili2025.0526.22
CLOBase Model=Llama-2-7B, Training Data=English + Swahili2025.0524.09
SFT-tgtBase Model=Llama-2-13B, Training Data=Target language only2025.0522.56
SFT-tgtBase Model=Llama-2-7B, Training Data=Target language only2025.0522.46
SFTBase Model=Llama-2-13B, Training Data=English + Swahili2025.0521.38
SFT+DPOBase Model=Llama-2-7B, Training Data=English + Swahili2025.0519.96
SFT+DPOBase Model=Llama-2-13B, Training Data=English + Swahili2025.0519.83
SFTBase Model=Llama-2-7B, Training Data=English + Swahili2025.0517.21