Multilingual Machine Translation on FLORES+ (devtest)

47.42spBLEUGoogle Translate

Evaluation Results

Method
Google Translate2026.02	47.42	89.42
Gemini 3 Pro2026.02	46.44	89.44
Gemini 2.5 Pro2026.02	46.13	89.38
Gemma3-12B2026.02	44.34	88.73
GPT-52026.02	43.64	89.19
NLLB-54.5B2026.02	43.23	88.1
Gemma2-9B2026.02	43.08	88.26
Google Translate2026.02	42.9	89.86
Gemini 3 Pro2026.02	42.42	90.35
Qwen3-14B2026.02	42.05	88.21
Gemini 2.5 Pro2026.02	41.15	90.07
Gemma3-4B2026.02	40.01	87.61
Qwen3-8B2026.02	39.59	87.6
Qwen2.5-14B2026.02	39.52	87.07
GPT-52026.02	38.42	89.86
Gemma3-12B2026.02	38.25	88.66
NLLB-54.5B2026.02	38.05	87.89
Qwen3-4B2026.02	36.72	86.57
Qwen2.5-7B2026.02	35.01	84.96
Gemma2-2B2026.02	34.78	84.87
Gemma2-9B2026.02	33.7	86.23
Gemma3-4B2026.02	31.88	86.21
Qwen3-14B2026.02	31.19	85.52
Gemma3-1B2026.02	30.74	84.17
Google Translate2026.02	30.74	87.46
Qwen3-1.7B2026.02	30.32	83.52
Gemini 3 Pro2026.02	29.9	87.97
Qwen2.5-3B2026.02	29.83	82.23
Gemini 2.5 Pro2026.02	29.12	87.76
Qwen3-8B2026.02	27.93	83.34
GPT-52026.02	26.36	87.46
Gemma3-12B2026.02	25.85	86.03
Qwen2.5-14B2026.02	25.32	79.03
NLLB-54.5B2026.02	25.17	85.27
Qwen2.5-1.5B2026.02	23.89	77.74
Qwen3-4B2026.02	22.88	78.55
Gemma2-2B2026.02	22.36	76.72
Gemma2-9B2026.02	21.02	82.6
Qwen3-14B2026.02	20.95	83.14
Qwen2.5-7B2026.02	20.4	72.85
Qwen3-0.6B2026.02	20.14	75.45
Gemma3-4B2026.02	19.88	83.05
Gemma3-1B2026.02	19.85	76.87
Qwen3-8B2026.02	17.88	80.43
Qwen2.5-14B2026.02	17.01	76.64
Qwen3-1.7B2026.02	15.55	68.96
Qwen2.5-3B2026.02	14.76	65.36
Qwen3-4B2026.02	14.61	75.82
Qwen2.5-7B2026.02	13.34	70.71
Qwen2.5-0.5B2026.02	12.9	66.08
Gemma2-2B2026.02	12.07	72.28
Qwen2.5-1.5B2026.02	10.77	58.6
Gemma3-1B2026.02	10.46	72.77
Qwen2.5-3B2026.02	9.14	62.85
Qwen3-1.7B2026.02	9.06	65.85
Gemma3-270M2026.02	9.02	66.35
Qwen3-0.6B2026.02	7.84	54.86
Qwen2.5-1.5B2026.02	6.54	56.68
Qwen2.5-0.5B2026.02	4.86	47.39
Gemma3-270M2026.02	4.68	53.47
Qwen3-0.6B2026.02	4.03	51.59
Qwen2.5-0.5B2026.02	2.66	46.41
Gemma3-270M2026.02	1.78	49.67