Machine Translation on FLORES+ (test)
57.64spBLEUMiLMMT-12B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MiLMMT-12BDirection=en → fr2026.02 | 57.64 | 89.39 | |
| MiLMMT-12BDirection=fr → en2026.02 | 51.6 | 89.88 | |
| MiLMMT-12BDirection=de → en2026.02 | 51.28 | 90.04 | |
| MiLMMT-12BDirection=ar → en2026.02 | 48.92 | 88.78 | |
| MiLMMT-12BDirection=en → de2026.02 | 48.91 | 89.25 | |
| Google TranslateLanguage Coverage=46 languages2026.02 | 47.42 | 89.42 | |
| MiLMMT-46-12BLanguage Coverage=26 languages2026.02 | 46.77 | 89.47 | |
| Gemini 3 ProLanguage Coverage=46 languages2026.02 | 46.44 | 89.44 | |
| Gemini 2.5 ProLanguage Coverage=46 languages2026.02 | 46.13 | 89.38 | |
| MiLMMT-46-12BLanguage Coverage=21 languages2026.02 | 45.96 | 89.4 | |
| MiLMMT-46-4BLanguage Coverage=26 languages2026.02 | 45.49 | 89.22 | |
| Seed-X-PPO-7BLanguage Coverage=26 languages2026.02 | 45.48 | 90.76 | |
| MiLMMT-46-12BLanguage Coverage=46 languages2026.02 | 45.45 | 89.22 | |
| Tower-Plus-9BLanguage Coverage=21 languages2026.02 | 45.32 | 89.35 | |
| GemmaX2-28-9BLanguage Coverage=28 languages2026.02 | 45.09 | 88.96 | |
| Google TranslateDirection=en → ar2026.02 | 44.9 | 88.92 | |
| MiLMMT-46-12BLanguage Coverage=28 languages2026.02 | 44.87 | 89.12 | |
| MiLMMT-46-4BLanguage Coverage=21 languages2026.02 | 44.69 | 89.16 | |
| Seed-X-Instruct-7BLanguage Coverage=26 languages2026.02 | 44.54 | 88.66 | |
| MiLMMT-46-12BLanguage Coverage=26 languages2026.02 | 44.49 | 90.91 | |
| Seed-X-Instruct-7BLanguage Coverage=26 languages2026.02 | 44.16 | 90.42 | |
| Seed-X-PPO-7BLanguage Coverage=26 languages2026.02 | 44.12 | 89.18 | |
| MiLMMT-46-12BLanguage Coverage=31 languages2026.02 | 44.05 | 89.01 | |
| MiLMMT-46-4BLanguage Coverage=46 languages2026.02 | 43.93 | 88.91 | |
| GPT-5Language Coverage=46 languages2026.02 | 43.64 | 89.19 | |
| Gemini 3 ProDirection=en → ar2026.02 | 43.6 | 88.6 | |
| MiLMMT-46-12BLanguage Coverage=21 languages2026.02 | 43.51 | 90.48 | |
| MiLMMT-46-4BLanguage Coverage=28 languages2026.02 | 43.34 | 88.81 | |
| Tower-Plus-9BLanguage Coverage=21 languages2026.02 | 43.33 | 90.31 | |
| NLLB-54.5BLanguage Coverage=46 languages2026.02 | 43.23 | 88.1 | |
| NLLB-54.5BDirection=en → ar2026.02 | 43.03 | 87.22 | |
| MiLMMT-46-4BLanguage Coverage=26 languages2026.02 | 42.91 | 90.49 | |
| Google TranslateLanguage Coverage=46 languages2026.02 | 42.9 | 89.86 | |
| MiLMMT-12BDirection=en → ar2026.02 | 42.61 | 88.35 | |
| Tower-Plus-2BLanguage Coverage=21 languages2026.02 | 42.54 | 88.77 | |
| MiLMMT-46-4BLanguage Coverage=31 languages2026.02 | 42.5 | 88.69 | |
| Gemini 3 ProLanguage Coverage=46 languages2026.02 | 42.42 | 90.35 | |
| GemmaX2-28-2BLanguage Coverage=28 languages2026.02 | 42.18 | 88.34 | |
| MiLMMT-46-1BLanguage Coverage=21 languages2026.02 | 42.17 | 88.53 | |
| MiLMMT-46-4BLanguage Coverage=21 languages2026.02 | 41.9 | 90.05 | |
| Gemini 2.5 ProDirection=en → ar2026.02 | 41.55 | 88.18 | |
| MiLMMT-46-12BLanguage Coverage=46 languages2026.02 | 41.24 | 90.16 | |
| Gemini 2.5 ProLanguage Coverage=46 languages2026.02 | 41.15 | 90.07 | |
| MiLMMT-46-1BLanguage Coverage=46 languages2026.02 | 40.61 | 88.04 | |
| Tower-Plus-2BLanguage Coverage=21 languages2026.02 | 40.41 | 89.41 | |
| MiLMMT-4BDirection=en → ar2026.02 | 40.31 | 88.01 | |
| MiLMMT-46-12BLanguage Coverage=28 languages2026.02 | 40.18 | 89.3 | |
| MiLMMT-46-1BLanguage Coverage=28 languages2026.02 | 39.85 | 87.92 | |
| GemmaX2-28-9BLanguage Coverage=28 languages2026.02 | 39.77 | 88.36 | |
| MiLMMT-46-12BLanguage Coverage=31 languages2026.02 | 39.64 | 89.58 | |
| MiLMMT-46-4BLanguage Coverage=46 languages2026.02 | 39.56 | 89.7 | |
| MiLMMT-46-1BLanguage Coverage=31 languages2026.02 | 39 | 87.79 | |
| MiLMMT-12BDirection=en → zhs2026.02 | 38.98 | 89.54 | |
| GPT-5Direction=en → ar2026.02 | 38.68 | 88.16 | |
| MiLMMT-46-4BLanguage Coverage=28 languages2026.02 | 38.52 | 88.86 | |
| GPT-5Language Coverage=46 languages2026.02 | 38.42 | 89.86 | |
| MiLMMT-46-1BLanguage Coverage=21 languages2026.02 | 38.14 | 88.71 | |
| NLLB-54.5BLanguage Coverage=46 languages2026.02 | 38.05 | 87.89 | |
| MiLMMT-46-4BLanguage Coverage=31 languages2026.02 | 37.96 | 89.12 | |
| GemmaX2-28-2BLanguage Coverage=28 languages2026.02 | 37.05 | 87.56 | |
| MiLMMT-12BDirection=zhs → en2026.02 | 36.6 | 88.11 | |
| Google Translatetype=Supervised NMT2026.02 | 36.08 | 88.24 | |
| TranslateGemma-12BLanguage Coverage=46 languages2026.02 | 35.45 | 88.09 | |
| MiLMMT-12BDirection=ja → en2026.02 | 35.36 | 88.97 | |
| MiLMMT-46-1BLanguage Coverage=46 languages2026.02 | 35.07 | 88.17 | |
| MiLMMT-1BDirection=en → ar2026.02 | 34.99 | 86.22 | |
| MiLMMT-12BDirection=en → ja2026.02 | 34.9 | 92.36 | |
| MiLMMT-46-1BLanguage Coverage=28 languages2026.02 | 34.17 | 87.37 | |
| Gemini 3 Proshots=82026.02 | 33.81 | 88.13 | |
| MiLMMT-46-1BLanguage Coverage=31 languages2026.02 | 33.6 | 87.66 | |
| TranslateGemma-4BLanguage Coverage=46 languages2026.02 | 33.42 | 87.26 | |
| Gemini 2.5 Proshots=82026.02 | 33.07 | 88.01 | |
| HY-MT1.5-7BLanguage Coverage=31 languages2026.02 | 31.68 | 87.34 | |
| GPT-5shots=82026.02 | 31.34 | 87.66 | |
| Gemma3-12Bshots=8, parameters=12B2026.02 | 31.13 | 86.84 | |
| TranslateGemma-12BLanguage Coverage=46 languages2026.02 | 31.05 | 89.08 | |
| Qwen3-14Bshots=8, parameters=14B2026.02 | 30.77 | 86.66 | |
| Google TranslateLanguage Coverage=46 languages2026.02 | 30.74 | 87.46 | |
| Hunyuan-MT-7BLanguage Coverage=31 languages2026.02 | 30.2 | 86.98 | |
| Gemini 3 ProLanguage Coverage=46 languages2026.02 | 29.9 | 87.97 | |
| MiLMMT-46-12BLanguage Coverage=26 languages2026.02 | 29.72 | 88.48 | |
| Seed-X-PPO-7BLanguage Coverage=26 languages2026.02 | 29.65 | 88.36 | |
| MiLMMT-46-12BLanguage Coverage=21 languages2026.02 | 29.43 | 88.14 | |
| Qwen3-8Bshots=8, parameters=8B2026.02 | 29.3 | 86.1 | |
| Qwen2.5-14Bshots=8, parameters=14B2026.02 | 29.28 | 85.64 | |
| Gemini 2.5 ProLanguage Coverage=46 languages2026.02 | 29.12 | 87.76 | |
| HY-MT1.5-7BLanguage Coverage=31 languages2026.02 | 28.97 | 87.45 | |
| Gemma2-9Bshots=8, parameters=9B2026.02 | 28.79 | 85.89 | |
| Seed-X-Instruct-7BLanguage Coverage=26 languages2026.02 | 28.6 | 87.53 | |
| MiLMMT-46-12BLanguage Coverage=46 languages2026.02 | 28.27 | 87.67 | |
| MiLMMT-46-12BLanguage Coverage=31 languages2026.02 | 28.12 | 87.14 | |
| MiLMMT-46-12BLanguage Coverage=28 languages2026.02 | 27.91 | 86.58 | |
| MiLMMT-46-4BLanguage Coverage=26 languages2026.02 | 27.89 | 87.94 | |
| Hunyuan-MT-7BLanguage Coverage=31 languages2026.02 | 27.84 | 87.14 | |
| TranslateGemma-4BLanguage Coverage=46 languages2026.02 | 27.71 | 85.09 | |
| MiLMMT-46-4BLanguage Coverage=21 languages2026.02 | 27.6 | 87.6 | |
| Tower-Plus-9BLanguage Coverage=21 languages2026.02 | 27.59 | 87.71 | |
| GemmaX2-28-9BLanguage Coverage=28 languages2026.02 | 27.48 | 85.71 | |
| Qwen3-4Bshots=8, parameters=4B2026.02 | 26.87 | 84.92 | |
| MiLMMT-46-4BLanguage Coverage=31 languages2026.02 | 26.42 | 86.63 |