Translation on FLORES-200 en-it (devtest)
33.831sacreBLEUGemmaX2-9B-b5
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GemmaX2-9B-b5Beam search=5 beams2026.01 | 33.831 | 61.671 | 0.1592 | 1.4747 | 0.8938 | 1.3952 | 0.8243 | |
| GemmaX2-9B2026.01 | 32.7799 | 60.8592 | 0.1501 | 1.4896 | 0.8924 | 1.3717 | 0.8229 | |
| GemmaX2-2B-b5Beam search=5 beams2026.01 | 31.9911 | 60.5123 | 0.1436 | 1.5573 | 0.8908 | 1.4123 | 0.8192 | |
| Madlad-7B2026.01 | 31.6561 | 60.2592 | 0.1462 | 1.7717 | 0.8847 | 1.7062 | 0.8027 | |
| Madlad-7B-b5Beam search=5 beams2026.01 | 31.5899 | 60.5317 | 0.152 | 1.7738 | 0.8845 | 1.6791 | 0.8038 | |
| Madlad-3B-b5Beam search=5 beams2026.01 | 31.4032 | 60.3812 | 0.1423 | 1.7752 | 0.8843 | 1.6775 | 0.8046 | |
| Madlad-3B2026.01 | 31.2632 | 60.096 | 0.1451 | 1.7913 | 0.8834 | 1.7193 | 0.8014 | |
| NLLB-3.3B-b5Beam search=5 beams2026.01 | 31.1904 | 59.8414 | 0.1402 | 1.7289 | 0.8853 | 1.684 | 0.8044 | |
| GemmaX2-2B2026.01 | 31.0059 | 59.5828 | 0.1385 | 1.7147 | 0.8869 | 1.5175 | 0.813 | |
| Tower-7B2026.01 | 30.4748 | 59.2816 | 0.1311 | 1.5994 | 0.8878 | 1.4263 | 0.8136 | |
| DIETA+allsynth2026.01 | 30.4376 | 59.5119 | 0.1323 | 2.0963 | 0.8751 | 1.9234 | 0.7855 | |
| NLLB-1.3B-b5Beam search=5 beams2026.01 | 30.1928 | 58.984 | 0.1287 | 1.8913 | 0.8804 | 1.9209 | 0.7895 | |
| NLLB-3.3B2026.01 | 30.0059 | 58.8228 | 0.1291 | 1.8871 | 0.8811 | 1.8405 | 0.7943 | |
| DIETA2026.01 | 29.9191 | 59.0087 | 0.1267 | 2.1968 | 0.8733 | 2.0097 | 0.7806 | |
| DIETA+nosynth2026.01 | 29.7304 | 59.1183 | 0.131 | 2.2151 | 0.8725 | 1.9921 | 0.7813 | |
| DIETA+cont2026.01 | 29.7134 | 59.1339 | 0.1319 | 2.172 | 0.8736 | 1.9675 | 0.7829 | |
| OpusMT-Big-b5Beam search=5 beams2026.01 | 29.6024 | 59.0557 | 0.1207 | 2.3736 | 0.8694 | 2.1018 | 0.7776 | |
| OpusMT-Big2026.01 | 29.5443 | 59.0024 | 0.1195 | 2.3761 | 0.8694 | 2.0994 | 0.7775 | |
| NLLB-1.3B2026.01 | 29.3377 | 58.0065 | 0.1177 | 2.0982 | 0.874 | 2.1723 | 0.7756 | |
| DIETA+BT2026.01 | 28.5118 | 58.0666 | 0.1151 | 2.603 | 0.8622 | 2.358 | 0.764 | |
| ModItalia-9B2026.01 | 28.5071 | 57.4549 | 0.1033 | 2.0779 | 0.8749 | 1.8068 | 0.7786 | |
| NLLB-600M-b5Beam search=5 beams2026.01 | 27.9796 | 57.5369 | 0.1136 | 2.3623 | 0.8689 | 2.2873 | 0.7717 | |
| OpusMT-b5Beam search=5 beams2026.01 | 27.6394 | 57.6967 | 0.1084 | 3.0737 | 0.8519 | 2.6895 | 0.7483 | |
| OpusMT2026.01 | 27.533 | 57.6113 | 0.1073 | 3.0805 | 0.8522 | 2.6936 | 0.7487 | |
| Llama3.1-8B-ITA2026.01 | 27.4665 | 57.0228 | 0.1046 | 2.1194 | 0.8768 | 1.8497 | 0.7921 | |
| NLLB-600M2026.01 | 26.8755 | 56.2636 | 0.0999 | 2.7228 | 0.8598 | 2.6371 | 0.7512 | |
| Cerbero-7B2026.01 | 25.6956 | 55.1158 | 0.0779 | 2.6306 | 0.8627 | 2.2672 | 0.7671 | |
| EuroLLM-9B2026.01 | 24.6029 | 54.6447 | 0.0682 | 2.0073 | 0.8728 | 1.6885 | 0.7847 | |
| PhiMaestra-32026.01 | 24.5784 | 54.5943 | 0.0758 | 2.785 | 0.862 | 2.3465 | 0.7647 | |
| Minerva-7B2026.01 | 24.3776 | 55.1011 | 0.0555 | 2.3166 | 0.8691 | 1.9943 | 0.7694 | |
| mBART502026.01 | 23.9405 | 54.2553 | 0.0731 | 3.395 | 0.85 | 2.774 | 0.7533 | |
| mBART50-b5Beam search=5 beams2026.01 | 23.9405 | 54.2553 | 0.0731 | 3.395 | 0.85 | 2.774 | 0.7533 | |
| Maestrale-v0.42026.01 | 23.4285 | 55.3653 | 0.0804 | 2.0409 | 0.8758 | 1.7759 | 0.7896 | |
| LLaMAntino-8B2026.01 | 23.137 | 53.482 | 0.0542 | 3.0499 | 0.8512 | 2.5582 | 0.75 | |
| EuroLLM-1.7B2026.01 | 19.0987 | 50.1396 | 0.0047 | 3.4103 | 0.8362 | 2.7074 | 0.7054 |