Multitask Language Understanding on MMMLU Korean 1.0 (test)
41.94AccuracyCLO
Evaluation Results
| Method | Links | |
|---|---|---|
| CLOBase Model=Qwen2.5-3B, Training Data=English + Korean2025.05 | 41.94 | |
| CLOBase Model=Llama-2-13B, Training Data=English + Korean2025.05 | 39.7 | |
| SFT-tgtBase Model=Llama-2-13B, Training Data=Target language only2025.05 | 36.8 | |
| SFTBase Model=Qwen2.5-3B, Training Data=English + Korean2025.05 | 35.9 | |
| SFTBase Model=Llama-2-13B, Training Data=English + Korean2025.05 | 34.39 | |
| CLOBase Model=Llama-3-8B, Training Data=English + Korean2025.05 | 32.73 | |
| SFT-tgtBase Model=Llama-3-8B, Training Data=Target language only2025.05 | 29.61 | |
| CLOBase Model=Llama-2-7B, Training Data=English + Korean2025.05 | 29.09 | |
| CLOBase Model=Mistral-7B, Training Data=English + Korean2025.05 | 28.31 | |
| SFT+DPOBase Model=Llama-2-7B, Training Data=English + Korean2025.05 | 28 | |
| SFT-tgtBase Model=Mistral-7B, Training Data=Target language only2025.05 | 27.65 | |
| SFT+DPOBase Model=Llama-3-8B, Training Data=English + Korean2025.05 | 27.48 | |
| SFT+DPOBase Model=Llama-2-13B, Training Data=English + Korean2025.05 | 26.79 | |
| SFT+DPOBase Model=Mistral-7B, Training Data=English + Korean2025.05 | 26.77 | |
| SFTBase Model=Mistral-7B, Training Data=English + Korean2025.05 | 25.94 | |
| SFT-tgtBase Model=Llama-2-7B, Training Data=Target language only2025.05 | 25.31 | |
| SFTBase Model=Llama-3-8B, Training Data=English + Korean2025.05 | 25.31 | |
| SFTBase Model=Llama-2-7B, Training Data=English + Korean2025.05 | 23.47 |