Machine Translation on Wikinews-25 en->it
50.6089sacreBLEUGemma-9B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma-9Bbeam_search=5 beams2026.01 | 50.6089 | 70.963 | 0.3428 | 2.3901 | 0.8709 | 2.6406 | 0.7275 | |
| Madlad-7Bbeam_search=5 beams2026.01 | 49.6611 | 70.4909 | 0.3467 | 2.9493 | 0.8527 | 3.0788 | 0.6962 | |
| Gemma-9B2026.01 | 49.5163 | 70.1275 | 0.3248 | 2.4948 | 0.8651 | 2.6346 | 0.7201 | |
| Madlad-3Bbeam_search=5 beams2026.01 | 49.227 | 69.9151 | 0.3253 | 3.1583 | 0.8481 | 3.1822 | 0.6895 | |
| Madlad-3B2026.01 | 49.1151 | 69.7597 | 0.3312 | 3.0873 | 0.8481 | 3.1496 | 0.6899 | |
| Madlad-7B2026.01 | 48.8297 | 69.8135 | 0.3382 | 2.9507 | 0.8539 | 3.014 | 0.6989 | |
| NLLB-3.3Bbeam_search=5 beams2026.01 | 48.2346 | 69.5857 | 0.3333 | 2.9385 | 0.8539 | 3.0433 | 0.7008 | |
| NLLB-1.3Bbeam_search=5 beams2026.01 | 47.8163 | 69.2843 | 0.3264 | 2.9598 | 0.8517 | 2.9972 | 0.6992 | |
| Gemma-2Bbeam_search=5 beams2026.01 | 47.8103 | 69.1842 | 0.315 | 2.676 | 0.8608 | 2.7424 | 0.7139 | |
| NLLB-3.3B2026.01 | 47.7769 | 68.9295 | 0.3207 | 3.1084 | 0.849 | 3.1482 | 0.6918 | |
| opus-mt-big2026.01 | 46.4855 | 68.013 | 0.3046 | 3.9025 | 0.8216 | 3.8201 | 0.6508 | |
| opus-mt-bigbeam_search=5 beams2026.01 | 46.3196 | 67.9282 | 0.3032 | 3.9616 | 0.82 | 3.8689 | 0.6479 | |
| NLLB-1.3B2026.01 | 46.306 | 67.9392 | 0.3027 | 3.1783 | 0.8445 | 3.2132 | 0.6867 | |
| DIETA_cont2026.01 | 46.0306 | 67.6836 | 0.2899 | 3.7464 | 0.8279 | 3.6355 | 0.6565 | |
| DIETA_nosynth2026.01 | 45.8281 | 67.8261 | 0.2945 | 3.7538 | 0.8272 | 3.7267 | 0.6543 | |
| DIETA2026.01 | 45.6901 | 67.5212 | 0.2955 | 3.7397 | 0.8309 | 3.6571 | 0.6591 | |
| DIETA_allsynth2026.01 | 45.6556 | 67.5683 | 0.2956 | 3.7476 | 0.8259 | 3.681 | 0.6532 | |
| Gemma-2B2026.01 | 45.3212 | 67.4632 | 0.2878 | 2.7597 | 0.8574 | 2.7503 | 0.7054 | |
| Tower-7B2026.01 | 44.7598 | 67.0473 | 0.2924 | 2.5816 | 0.8589 | 2.6614 | 0.7095 | |
| NLLB-600Mbeam_search=5 beams2026.01 | 44.319 | 67.1346 | 0.2936 | 3.5221 | 0.8371 | 3.4784 | 0.6737 | |
| NLLB-600M2026.01 | 43.2321 | 65.9354 | 0.2805 | 3.7766 | 0.8258 | 3.6958 | 0.655 | |
| DIETA_BT2026.01 | 43.0851 | 65.8102 | 0.2765 | 4.3233 | 0.8141 | 4.2341 | 0.6253 | |
| opus-mt2026.01 | 40.9083 | 64.8212 | 0.2623 | 4.8126 | 0.8017 | 4.6008 | 0.6105 | |
| opus-mtbeam_search=5 beams2026.01 | 40.6303 | 64.7364 | 0.2596 | 4.8213 | 0.8008 | 4.6089 | 0.611 | |
| ModelloItalia-9B2026.01 | 40.4526 | 63.7778 | 0.2324 | 3.3922 | 0.8363 | 3.1609 | 0.6588 | |
| Llama-3.1-8B2026.01 | 38.9129 | 63.2285 | 0.236 | 3.3564 | 0.836 | 3.2611 | 0.6701 | |
| mBART2026.01 | 36.6504 | 61.5672 | 0.2249 | 4.238 | 0.8163 | 3.8689 | 0.6423 | |
| mBARTbeam_search=5 beams2026.01 | 36.6504 | 61.5672 | 0.2249 | 4.238 | 0.8163 | 3.8689 | 0.6423 | |
| PhiMaestra-32026.01 | 35.7865 | 61.2021 | 0.2205 | 4.3248 | 0.8099 | 4.0798 | 0.6279 | |
| Cerbero-7B2026.01 | 35.3794 | 60.5522 | 0.1616 | 4.2303 | 0.8108 | 4.0116 | 0.6262 | |
| Minerva-7B2026.01 | 32.3341 | 60.1072 | 0.1808 | 3.6852 | 0.8275 | 3.5744 | 0.6457 | |
| EuroLLM-9B2026.01 | 30.7073 | 58.4158 | 0.153 | 3.3383 | 0.8275 | 2.9672 | 0.6693 | |
| Maestrale-v0.42026.01 | 29.6953 | 58.4669 | 0.1675 | 3.2506 | 0.829 | 2.999 | 0.6689 | |
| LLaMAntino-8B2026.01 | 29.4182 | 56.4517 | 0.1041 | 4.5677 | 0.7942 | 4.2699 | 0.6038 | |
| EuroLLM-1.7B2026.01 | 21.6238 | 51.2821 | 0.0725 | 5.1299 | 0.7753 | 4.4343 | 0.5663 |