Machine Translation on NTREX it->en 128 (test)
49.2683sacreBLEUGemma-9B-b5
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma-9B-b5Beam search=5 beams2026.01 | 49.2683 | 70.3634 | 0.4787 | 2.4325 | 0.8888 | 2.5363 | 0.7695 | |
| Gemma-9BBeam search=default2026.01 | 48.1639 | 69.7526 | 0.4703 | 2.4812 | 0.8874 | 2.5546 | 0.7693 | |
| Gemma-2B-b5Beam search=5 beams2026.01 | 47.159 | 68.6866 | 0.4527 | 2.5509 | 0.8851 | 2.4879 | 0.7723 | |
| Gemma-2BBeam search=default2026.01 | 46.5254 | 68.5879 | 0.4485 | 2.6064 | 0.8847 | 2.5327 | 0.7708 | |
| Tower-7BBeam search=default2026.01 | 45.7063 | 68.1702 | 0.4463 | 2.6006 | 0.8834 | 2.5247 | 0.7698 | |
| NLLB-3.3B-b5Beam search=5 beams2026.01 | 44.0335 | 67.6084 | 0.434 | 2.6609 | 0.878 | 2.4876 | 0.7727 | |
| NLLB-3.3BBeam search=default2026.01 | 43.7135 | 67.272 | 0.4306 | 2.7114 | 0.8774 | 2.5379 | 0.7698 | |
| NLLB-1.3B-b5Beam search=5 beams2026.01 | 43.0356 | 66.8704 | 0.4267 | 2.7521 | 0.8768 | 2.5401 | 0.7707 | |
| DIETA+contBeam search=default2026.01 | 42.7624 | 66.3234 | 0.4121 | 2.9645 | 0.8747 | 2.7531 | 0.7604 | |
| Madlad-7BBeam search=default2026.01 | 42.5244 | 66.6484 | 0.4369 | 2.6337 | 0.8811 | 2.4991 | 0.7736 | |
| DIETA+allsynthBeam search=default2026.01 | 42.505 | 66.6534 | 0.4177 | 2.9195 | 0.8763 | 2.7389 | 0.7611 | |
| Madlad-7B-b5Beam search=5 beams2026.01 | 42.2828 | 66.7757 | 0.4367 | 2.5962 | 0.8812 | 2.4546 | 0.7757 | |
| NLLB-1.3BBeam search=default2026.01 | 42.2195 | 66.3278 | 0.4197 | 2.8303 | 0.8754 | 2.6013 | 0.768 | |
| Madlad-3BBeam search=default2026.01 | 41.7829 | 66.0737 | 0.4264 | 2.7272 | 0.879 | 2.5132 | 0.7734 | |
| Madlad-3B-b5Beam search=5 beams2026.01 | 41.5983 | 66.2366 | 0.4246 | 2.6997 | 0.8785 | 2.4917 | 0.7744 | |
| opus-mt-big-b5Beam search=5 beams2026.01 | 41.5459 | 65.7754 | 0.4063 | 2.9981 | 0.8718 | 2.7196 | 0.7652 | |
| opus-mt-bigBeam search=default2026.01 | 41.5136 | 65.7436 | 0.4059 | 3.0061 | 0.872 | 2.7228 | 0.765 | |
| Llama-3.1-8BBeam search=default2026.01 | 41.2522 | 64.7492 | 0.2846 | 3.4054 | 0.872 | 3.1498 | 0.7194 | |
| DIETA+BTBeam search=default2026.01 | 41.1467 | 65.1165 | 0.3777 | 3.2974 | 0.8664 | 2.9899 | 0.7499 | |
| NLLB-600M-b5Beam search=5 beams2026.01 | 40.6537 | 65.1725 | 0.3949 | 3.0685 | 0.8679 | 2.7057 | 0.7632 | |
| NLLB-600MBeam search=default2026.01 | 40.0278 | 64.7658 | 0.3855 | 3.1761 | 0.8655 | 2.7883 | 0.7583 | |
| DIETA+nosynthBeam search=default2026.01 | 39.2049 | 64.7584 | 0.3929 | 3.0519 | 0.8709 | 2.7792 | 0.7585 | |
| DIETABeam search=default2026.01 | 38.983 | 64.4056 | 0.3885 | 3.117 | 0.8691 | 2.8312 | 0.7561 | |
| PhiMaestra-3Beam search=default2026.01 | 36.5609 | 62.7982 | 0.3538 | 3.2044 | 0.8635 | 2.8676 | 0.7534 | |
| opus-mtBeam search=default2026.01 | 36.0435 | 62.7542 | 0.3461 | 3.428 | 0.8565 | 2.9983 | 0.754 | |
| opus-mt-b5Beam search=5 beams2026.01 | 36.008 | 62.7413 | 0.3458 | 3.4471 | 0.8563 | 3.008 | 0.7537 | |
| mBARTBeam search=default2026.01 | 34.9348 | 61.1602 | 0.3029 | 3.991 | 0.8479 | 3.3345 | 0.7294 | |
| mBART-b5Beam search=5 beams2026.01 | 34.9348 | 61.1602 | 0.3029 | 3.991 | 0.8479 | 3.3345 | 0.7294 | |
| ModelloItalia-9BBeam search=default2026.01 | 34.6847 | 61.3331 | 0.2572 | 3.649 | 0.8548 | 3.1396 | 0.7192 | |
| LLaMAntino-8BBeam search=default2026.01 | 34.044 | 62.1239 | 0.33 | 3.1905 | 0.8589 | 2.8648 | 0.7529 | |
| Maestrale-v0.4Beam search=default2026.01 | 32.4728 | 60.157 | 0.2952 | 3.2991 | 0.8585 | 2.9898 | 0.7362 | |
| EuroLLM-9BBeam search=default2026.01 | 32.2015 | 60.8868 | 0.3495 | 3.092 | 0.8636 | 2.7051 | 0.7557 | |
| Cerbero-7BBeam search=default2026.01 | 30.976 | 57.1923 | 0.0215 | 4.7368 | 0.8467 | 4.0855 | 0.6427 | |
| EuroLLM-1.7BBeam search=default2026.01 | 26.4106 | 56.7543 | 0.1282 | 4.4232 | 0.8299 | 3.7631 | 0.7023 | |
| Minerva-7BBeam search=default2026.01 | 25.7506 | 52.6011 | -0.2292 | 7.1023 | 0.7727 | 6.6963 | 0.5846 |