Machine Translation on WMT RU-EN (test)
22.2BLEUMix-MoE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mix-MoEBase Model=Llama3.2-1B, Training Strategy=Mix-MoE2026.05 | 22.2 | 44.6 | 82.1 | |
| Qwen2.5-1.5BBase Model=Qwen2.5-1.5B2026.05 | 21.8 | 44.1 | 79.8 | |
| MOE-LPRBase Model=Llama3.2-1B, Training Strategy=MOE-LPR2026.05 | 21.5 | 43.5 | 81.4 | |
| MoEBase Model=Llama3.2-1B, Training Strategy=MoE2026.05 | 19 | 40.4 | 74.9 | |
| Full FinetuneBase Model=Llama3.2-1B, Training Strategy=Full Finetune2026.05 | 18.9 | 40.2 | 74.4 | |
| Dense PT+SFTBase Model=Llama3.2-1B, Training Strategy=Dense PT+SFT2026.05 | 17.7 | 41.5 | 76.5 | |
| Qwen2.5-0.5BBase Model=Qwen2.5-0.5B2026.05 | 17.3 | 39.8 | 76.4 | |
| LLAMA-ProBase Model=Llama3.2-1B, Training Strategy=LLAMA-Pro2026.05 | 14.6 | 36 | 68 | |
| LoRABase Model=Llama3.2-1B, Training Strategy=LoRA2026.05 | 13.6 | 34.6 | 67.4 | |
| LLAMA-MOE v2Base Model=Llama3.2-1B, Training Strategy=LLAMA-MOE v22026.05 | 13.4 | 33 | 58.9 | |
| Llama3.2-1BBase Model=Llama3.2-1B2026.05 | 10.2 | 28.5 | 58.3 |