List your company
ResearchFormerly WizWand

Multitask Language Understanding on MMMLU Korean 1.0 (test)

41.94AccuracyCLO
22.731227.718132.70537.6919May 20, 2025
Updated 8mo ago

Evaluation results

MethodLinks
CLOBase Model=Qwen2.5-3B, Training Data=English + Korean2025.0541.94
CLOBase Model=Llama-2-13B, Training Data=English + Korean2025.0539.7
SFT-tgtBase Model=Llama-2-13B, Training Data=Target language only2025.0536.8
SFTBase Model=Qwen2.5-3B, Training Data=English + Korean2025.0535.9
SFTBase Model=Llama-2-13B, Training Data=English + Korean2025.0534.39
CLOBase Model=Llama-3-8B, Training Data=English + Korean2025.0532.73
SFT-tgtBase Model=Llama-3-8B, Training Data=Target language only2025.0529.61
CLOBase Model=Llama-2-7B, Training Data=English + Korean2025.0529.09
CLOBase Model=Mistral-7B, Training Data=English + Korean2025.0528.31
SFT+DPOBase Model=Llama-2-7B, Training Data=English + Korean2025.0528
SFT-tgtBase Model=Mistral-7B, Training Data=Target language only2025.0527.65
SFT+DPOBase Model=Llama-3-8B, Training Data=English + Korean2025.0527.48
SFT+DPOBase Model=Llama-2-13B, Training Data=English + Korean2025.0526.79
SFT+DPOBase Model=Mistral-7B, Training Data=English + Korean2025.0526.77
SFTBase Model=Mistral-7B, Training Data=English + Korean2025.0525.94
SFT-tgtBase Model=Llama-2-7B, Training Data=Target language only2025.0525.31
SFTBase Model=Llama-3-8B, Training Data=English + Korean2025.0525.31
SFTBase Model=Llama-2-7B, Training Data=English + Korean2025.0523.47