Language Understanding on MMLU en
70.23MMLU (en)Qwen2-FFT
Evaluation results
| Method | Links | |
|---|---|---|
| Qwen2-FFTTarget Language Adaptation Setting=Korean, CP / ELO pretraining Data size (Tokens)=10 GB (3.1 B), CP / ELO pretraining Time=20.5 h, Total training time=20.5 h2026.01 | 70.23 | |
| Qwen2-ELOTarget Language Adaptation Setting=Korean, CP / ELO pretraining Data size (Tokens)=9 GB (2.8 B), CP / ELO pretraining Time=1.8 h, Layer alignment Data size (Tokens)=1 GB (0.3 B), Layer alignment Time=2.1 h, Total training time=3.9 h2026.01 | 70.11 | |
| Qwen2-7B-InstructTarget Language Adaptation Setting=Korean2026.01 | 69.89 | |
| Llama3.1-8B-InstructTarget Language Adaptation Setting=Korean2026.01 | 68.07 | |
| Llama3.1-8B-InstructTarget Language Adaptation Setting=Japanese2026.01 | 68.07 | |
| Llama3.1-FFTTarget Language Adaptation Setting=Korean, CP / ELO pretraining Data size (Tokens)=10 GB (2.8 B), CP / ELO pretraining Time=19.8 h, Total training time=19.8 h2026.01 | 67.51 | |
| Llama3.1-ELOTarget Language Adaptation Setting=Japanese, CP / ELO pretraining Data size (Tokens)=9 GB (2.4 B), CP / ELO pretraining Time=1.4 h, Layer alignment Data size (Tokens)=1 GB (0.3 B), Layer alignment Time=2.0 h, Total training time=3.4 h2026.01 | 67.51 | |
| Llama3.1-FFTTarget Language Adaptation Setting=Japanese, CP / ELO pretraining Data size (Tokens)=10 GB (2.7 B), CP / ELO pretraining Time=19.4 h, Total training time=19.4 h2026.01 | 67.5 | |
| Llama3.1-ELOTarget Language Adaptation Setting=Korean, CP / ELO pretraining Data size (Tokens)=9 GB (2.5 B), CP / ELO pretraining Time=1.5 h, Layer alignment Data size (Tokens)=1 GB (0.3 B), Layer alignment Time=2.0 h, Total training time=3.5 h2026.01 | 66.69 | |
| Mistral-7B-Instruct-v0.3Target Language Adaptation Setting=Korean2026.01 | 59.69 | |
| Mistral-7B-Instruct-v0.3Target Language Adaptation Setting=Japanese2026.01 | 59.69 | |
| Mistral-ELOTarget Language Adaptation Setting=Korean, CP / ELO pretraining Data size (Tokens)=9 GB (4.2 B), CP / ELO pretraining Time=1.7 h, Layer alignment Data size (Tokens)=1 GB (0.6 B), Layer alignment Time=3.1 h, Total training time=4.8 h2026.01 | 58.9 | |
| Mistral-FFTTarget Language Adaptation Setting=Korean, CP / ELO pretraining Data size (Tokens)=10 GB (4.7 B), CP / ELO pretraining Time=31.0 h, Total training time=31.0 h2026.01 | 57.47 | |
| DenseTarget Language=Turkish, Backbone=Llama-3.2-3B2026.03 | 56.45 | |
| DenseTarget Language=Hungarian, Backbone=Llama-3.2-3B2026.03 | 56.45 | |
| NeuronMoETarget Language=Turkish, Backbone=Llama-3.2-3B, Number of Experts=50 (-40%)2026.03 | 56.21 | |
| NeuronMoETarget Language=Hungarian, Backbone=Llama-3.2-3B, Number of Experts=47 (-44%)2026.03 | 56.1 | |
| LayerMoETarget Language=Turkish, Backbone=Llama-3.2-3B, Number of Experts=842026.03 | 56.05 | |
| LayerMoETarget Language=Hungarian, Backbone=Llama-3.2-3B, Number of Experts=842026.03 | 56.05 | |
| Mistral-ELOTarget Language Adaptation Setting=Japanese, CP / ELO pretraining Data size (Tokens)=9 GB (3.7 B), CP / ELO pretraining Time=1.7 h, Layer alignment Data size (Tokens)=1 GB (0.4 B), Layer alignment Time=3.0 h, Total training time=4.7 h2026.01 | 55.19 | |
| Mistral-FFTTarget Language Adaptation Setting=Japanese, CP / ELO pretraining Data size (Tokens)=10 GB (4.1 B), CP / ELO pretraining Time=29.7 h, Total training time=29.7 h2026.01 | 54.86 |