Machine Translation on NTREX (en->it) 128 (test)
50.7462sacreBLEUGemma-9B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma-9BBeam search=default2026.01 | 50.7462 | 71.7725 | 0.3523 | 1.6551 | 0.8992 | 1.8245 | 0.7912 | |
| Gemma-9B-b5Beam search=5 beams2026.01 | 50.4767 | 72.4682 | 0.3596 | 1.6006 | 0.901 | 1.7965 | 0.7933 | |
| Gemma-2B-b5Beam search=5 beams2026.01 | 45.7915 | 68.9844 | 0.2932 | 1.6976 | 0.8934 | 1.7898 | 0.7925 | |
| Gemma-2BBeam search=default2026.01 | 44.6901 | 68.0057 | 0.2778 | 1.8038 | 0.8902 | 1.8589 | 0.7861 | |
| Tower-7BBeam search=default2026.01 | 41.7372 | 66.0983 | 0.247 | 1.8635 | 0.884 | 1.8721 | 0.7857 | |
| Madlad-7B-b5Beam search=5 beams2026.01 | 38.9525 | 64.3319 | 0.2293 | 2.3629 | 0.8716 | 2.2685 | 0.7635 | |
| Madlad-3B-b5Beam search=5 beams2026.01 | 38.4722 | 64.0904 | 0.2255 | 2.4614 | 0.8687 | 2.3322 | 0.7608 | |
| Madlad-7BBeam search=default2026.01 | 38.4578 | 63.7821 | 0.2214 | 2.3396 | 0.8707 | 2.2737 | 0.7634 | |
| Madlad-3BBeam search=default2026.01 | 37.8887 | 63.4694 | 0.2181 | 2.4718 | 0.8687 | 2.3323 | 0.7598 | |
| NLLB-3.3B-b5Beam search=5 beams2026.01 | 37.4447 | 63.4329 | 0.2212 | 2.3616 | 0.8695 | 2.223 | 0.766 | |
| NLLB-1.3B-b5Beam search=5 beams2026.01 | 36.8762 | 63.1081 | 0.2096 | 2.4641 | 0.8663 | 2.2992 | 0.7648 | |
| NLLB-3.3BBeam search=default2026.01 | 36.5066 | 62.6141 | 0.2093 | 2.5183 | 0.8663 | 2.3542 | 0.7583 | |
| DIETA+contBeam search=default2026.01 | 36.3722 | 62.4029 | 0.2002 | 3.0613 | 0.8519 | 2.8206 | 0.7251 | |
| opus-mt-big-b5Beam search=5 beams2026.01 | 36.3222 | 62.4308 | 0.1966 | 3.3127 | 0.8432 | 3.0016 | 0.7158 | |
| ModelloItalia-9BBeam search=default2026.01 | 36.2878 | 62.0944 | 0.1864 | 2.4967 | 0.8628 | 2.3314 | 0.7418 | |
| opus-mt-bigBeam search=default2026.01 | 36.1768 | 62.2987 | 0.1968 | 3.3244 | 0.8428 | 3.0119 | 0.7156 | |
| DIETA+allsynthBeam search=default2026.01 | 36.0593 | 62.2428 | 0.1912 | 3.0298 | 0.8517 | 2.7831 | 0.7258 | |
| NLLB-1.3BBeam search=default2026.01 | 36.0274 | 62.2182 | 0.1985 | 2.6634 | 0.8617 | 2.4676 | 0.7532 | |
| DIETA+nosynthBeam search=default2026.01 | 35.9564 | 62.2259 | 0.1902 | 3.1924 | 0.8479 | 2.9463 | 0.7167 | |
| DIETABeam search=default2026.01 | 35.9073 | 62.1086 | 0.1926 | 3.1779 | 0.8487 | 2.929 | 0.7196 | |
| NLLB-600M-b5Beam search=5 beams2026.01 | 35.0643 | 61.8143 | 0.1968 | 2.9996 | 0.8546 | 2.712 | 0.7389 | |
| DIETA+BTBeam search=default2026.01 | 34.6548 | 60.8428 | 0.1746 | 3.4625 | 0.8396 | 3.1616 | 0.7046 | |
| NLLB-600MBeam search=default2026.01 | 34.2615 | 60.9701 | 0.186 | 3.2779 | 0.8466 | 2.9786 | 0.7233 | |
| opus-mt-b5Beam search=5 beams2026.01 | 32.7081 | 60.1931 | 0.169 | 4.1173 | 0.8215 | 3.7607 | 0.6765 | |
| opus-mtBeam search=default2026.01 | 32.6806 | 60.1638 | 0.1692 | 4.1174 | 0.822 | 3.7494 | 0.6762 | |
| Llama-3.1-8BBeam search=default2026.01 | 31.166 | 58.9875 | 0.1484 | 2.7722 | 0.8589 | 2.5072 | 0.751 | |
| Minerva-7BBeam search=default2026.01 | 30.2021 | 58.7382 | 0.132 | 2.8985 | 0.8528 | 2.6651 | 0.7286 | |
| Cerbero-7BBeam search=default2026.01 | 29.7079 | 57.3078 | 0.1096 | 3.5111 | 0.8362 | 3.2268 | 0.7057 | |
| mBARTBeam search=default2026.01 | 29.7014 | 57.4304 | 0.1415 | 4.1793 | 0.8268 | 3.6291 | 0.6878 | |
| mBART-b5Beam search=5 beams2026.01 | 29.7014 | 57.4304 | 0.1415 | 4.1793 | 0.8268 | 3.6291 | 0.6878 | |
| PhiMaestra-3Beam search=default2026.01 | 29.065 | 57.2235 | 0.1274 | 3.762 | 0.8336 | 3.3418 | 0.6969 | |
| EuroLLM-9BBeam search=default2026.01 | 27.0934 | 57.2185 | 0.1041 | 2.5383 | 0.856 | 2.2266 | 0.745 | |
| Maestrale-v0.4Beam search=default2026.01 | 26.4776 | 56.4607 | 0.1038 | 2.655 | 0.851 | 2.3368 | 0.7429 | |
| LLaMAntino-8BBeam search=default2026.01 | 24.7926 | 53.738 | 0.0606 | 3.9447 | 0.8198 | 3.5144 | 0.6846 | |
| EuroLLM-1.7BBeam search=default2026.01 | 20.4871 | 51.9333 | 0.0146 | 4.3742 | 0.8061 | 3.7145 | 0.6428 |