Multilingual Machine Translation on FLORES+ (devtest)
47.42spBLEUGoogle Translate
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Google Translatetype=Supervised NMT2026.02 | 47.42 | 89.42 | |
| Gemini 3 Proshots=82026.02 | 46.44 | 89.44 | |
| Gemini 2.5 Proshots=82026.02 | 46.13 | 89.38 | |
| Gemma3-12Bshots=8, parameters=12B2026.02 | 44.34 | 88.73 | |
| GPT-5shots=82026.02 | 43.64 | 89.19 | |
| NLLB-54.5Btype=Supervised NMT, parameters=54.5B2026.02 | 43.23 | 88.1 | |
| Gemma2-9Bshots=8, parameters=9B2026.02 | 43.08 | 88.26 | |
| Google Translatetype=Supervised NMT2026.02 | 42.9 | 89.86 | |
| Gemini 3 Proshots=82026.02 | 42.42 | 90.35 | |
| Qwen3-14Bshots=8, parameters=14B2026.02 | 42.05 | 88.21 | |
| Gemini 2.5 Proshots=82026.02 | 41.15 | 90.07 | |
| Gemma3-4Bshots=8, parameters=4B2026.02 | 40.01 | 87.61 | |
| Qwen3-8Bshots=8, parameters=8B2026.02 | 39.59 | 87.6 | |
| Qwen2.5-14Bshots=8, parameters=14B2026.02 | 39.52 | 87.07 | |
| GPT-5shots=82026.02 | 38.42 | 89.86 | |
| Gemma3-12Bshots=8, parameters=12B2026.02 | 38.25 | 88.66 | |
| NLLB-54.5Btype=Supervised NMT, parameters=54.5B2026.02 | 38.05 | 87.89 | |
| Qwen3-4Bshots=8, parameters=4B2026.02 | 36.72 | 86.57 | |
| Qwen2.5-7Bshots=8, parameters=7B2026.02 | 35.01 | 84.96 | |
| Gemma2-2Bshots=8, parameters=2B2026.02 | 34.78 | 84.87 | |
| Gemma2-9Bshots=8, parameters=9B2026.02 | 33.7 | 86.23 | |
| Gemma3-4Bshots=8, parameters=4B2026.02 | 31.88 | 86.21 | |
| Qwen3-14Bshots=8, parameters=14B2026.02 | 31.19 | 85.52 | |
| Gemma3-1Bshots=8, parameters=1B2026.02 | 30.74 | 84.17 | |
| Google Translatetype=Supervised NMT2026.02 | 30.74 | 87.46 | |
| Qwen3-1.7Bshots=8, parameters=1.7B2026.02 | 30.32 | 83.52 | |
| Gemini 3 Proshots=82026.02 | 29.9 | 87.97 | |
| Qwen2.5-3Bshots=8, parameters=3B2026.02 | 29.83 | 82.23 | |
| Gemini 2.5 Proshots=82026.02 | 29.12 | 87.76 | |
| Qwen3-8Bshots=8, parameters=8B2026.02 | 27.93 | 83.34 | |
| GPT-5shots=82026.02 | 26.36 | 87.46 | |
| Gemma3-12Bshots=8, parameters=12B2026.02 | 25.85 | 86.03 | |
| Qwen2.5-14Bshots=8, parameters=14B2026.02 | 25.32 | 79.03 | |
| NLLB-54.5Btype=Supervised NMT, parameters=54.5B2026.02 | 25.17 | 85.27 | |
| Qwen2.5-1.5Bshots=8, parameters=1.5B2026.02 | 23.89 | 77.74 | |
| Qwen3-4Bshots=8, parameters=4B2026.02 | 22.88 | 78.55 | |
| Gemma2-2Bshots=8, parameters=2B2026.02 | 22.36 | 76.72 | |
| Gemma2-9Bshots=8, parameters=9B2026.02 | 21.02 | 82.6 | |
| Qwen3-14Bshots=8, parameters=14B2026.02 | 20.95 | 83.14 | |
| Qwen2.5-7Bshots=8, parameters=7B2026.02 | 20.4 | 72.85 | |
| Qwen3-0.6Bshots=8, parameters=0.6B2026.02 | 20.14 | 75.45 | |
| Gemma3-4Bshots=8, parameters=4B2026.02 | 19.88 | 83.05 | |
| Gemma3-1Bshots=8, parameters=1B2026.02 | 19.85 | 76.87 | |
| Qwen3-8Bshots=8, parameters=8B2026.02 | 17.88 | 80.43 | |
| Qwen2.5-14Bshots=8, parameters=14B2026.02 | 17.01 | 76.64 | |
| Qwen3-1.7Bshots=8, parameters=1.7B2026.02 | 15.55 | 68.96 | |
| Qwen2.5-3Bshots=8, parameters=3B2026.02 | 14.76 | 65.36 | |
| Qwen3-4Bshots=8, parameters=4B2026.02 | 14.61 | 75.82 | |
| Qwen2.5-7Bshots=8, parameters=7B2026.02 | 13.34 | 70.71 | |
| Qwen2.5-0.5Bshots=8, parameters=0.5B2026.02 | 12.9 | 66.08 | |
| Gemma2-2Bshots=8, parameters=2B2026.02 | 12.07 | 72.28 | |
| Qwen2.5-1.5Bshots=8, parameters=1.5B2026.02 | 10.77 | 58.6 | |
| Gemma3-1Bshots=8, parameters=1B2026.02 | 10.46 | 72.77 | |
| Qwen2.5-3Bshots=8, parameters=3B2026.02 | 9.14 | 62.85 | |
| Qwen3-1.7Bshots=8, parameters=1.7B2026.02 | 9.06 | 65.85 | |
| Gemma3-270Mshots=8, parameters=270M2026.02 | 9.02 | 66.35 | |
| Qwen3-0.6Bshots=8, parameters=0.6B2026.02 | 7.84 | 54.86 | |
| Qwen2.5-1.5Bshots=8, parameters=1.5B2026.02 | 6.54 | 56.68 | |
| Qwen2.5-0.5Bshots=8, parameters=0.5B2026.02 | 4.86 | 47.39 | |
| Gemma3-270Mshots=8, parameters=270M2026.02 | 4.68 | 53.47 | |
| Qwen3-0.6Bshots=8, parameters=0.6B2026.02 | 4.03 | 51.59 | |
| Qwen2.5-0.5Bshots=8, parameters=0.5B2026.02 | 2.66 | 46.41 | |
| Gemma3-270Mshots=8, parameters=270M2026.02 | 1.78 | 49.67 |