Machine Translation on Wikinews-25 it->en
46.8309sacreBLEUGemma-9B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma-9B2026.01 | 46.8309 | 68.1128 | 0.4364 | 2.6548 | 0.8719 | 2.9823 | 0.7406 | |
| Gemma-9Bbeam_search=5 beams2026.01 | 46.3719 | 67.9598 | 0.4297 | 2.6547 | 0.8715 | 2.9497 | 0.742 | |
| Madlad-7B2026.01 | 44.7538 | 67.3971 | 0.4161 | 2.6853 | 0.8708 | 2.9593 | 0.7473 | |
| Madlad-7Bbeam_search=5 beams2026.01 | 44.4322 | 67.5081 | 0.4206 | 2.6612 | 0.8708 | 2.9019 | 0.748 | |
| Gemma-2B2026.01 | 44.1352 | 66.4546 | 0.4003 | 2.8834 | 0.8668 | 3.0933 | 0.7347 | |
| NLLB-3.3Bbeam_search=5 beams2026.01 | 44.1242 | 67.1718 | 0.4088 | 2.7849 | 0.8667 | 2.9833 | 0.7385 | |
| Gemma-2Bbeam_search=5 beams2026.01 | 43.9921 | 66.6708 | 0.4025 | 2.8107 | 0.8673 | 2.9989 | 0.7139 | |
| NLLB-1.3Bbeam_search=5 beams2026.01 | 43.9475 | 66.8214 | 0.4001 | 2.9083 | 0.8636 | 3.0729 | 0.7358 | |
| Tower-7B2026.01 | 43.9073 | 66.6963 | 0.4045 | 2.7601 | 0.868 | 2.9626 | 0.7412 | |
| DIETA_allsynth2026.01 | 43.8153 | 67.3398 | 0.4161 | 2.8457 | 0.8682 | 3.1184 | 0.7341 | |
| opus-mt-big2026.01 | 43.8037 | 67.0188 | 0.3969 | 3.0442 | 0.8643 | 3.1695 | 0.7319 | |
| opus-mt-bigbeam_search=5 beams2026.01 | 43.7779 | 66.9952 | 0.3961 | 3.0431 | 0.8643 | 3.1643 | 0.7317 | |
| Madlad-3Bbeam_search=5 beams2026.01 | 43.6775 | 67.0854 | 0.4017 | 2.7614 | 0.8676 | 2.9197 | 0.7454 | |
| NLLB-3.3B2026.01 | 43.6761 | 66.8444 | 0.3997 | 2.8293 | 0.8658 | 3.0276 | 0.7338 | |
| DIETA_cont2026.01 | 43.1714 | 66.6126 | 0.4064 | 2.8662 | 0.8654 | 3.1471 | 0.7353 | |
| NLLB-1.3B2026.01 | 42.5641 | 65.924 | 0.387 | 2.9223 | 0.8614 | 3.0845 | 0.7316 | |
| Madlad-3B2026.01 | 42.4652 | 66.6616 | 0.399 | 2.7922 | 0.8666 | 3.0253 | 0.7428 | |
| DIETA2026.01 | 41.7966 | 65.6442 | 0.3996 | 2.9451 | 0.8639 | 3.0952 | 0.7321 | |
| NLLB-600Mbeam_search=5 beams2026.01 | 41.5714 | 65.0993 | 0.36 | 3.1482 | 0.8547 | 3.1991 | 0.7277 | |
| DIETA_nosynth2026.01 | 41.5344 | 65.6032 | 0.3907 | 2.9653 | 0.8621 | 3.1194 | 0.7321 | |
| DIETA_BT2026.01 | 41.4561 | 64.9263 | 0.3652 | 3.2289 | 0.8565 | 3.3888 | 0.7226 | |
| NLLB-600M2026.01 | 40.5977 | 64.28 | 0.3441 | 3.2964 | 0.8515 | 3.3083 | 0.7244 | |
| opus-mtbeam_search=5 beams2026.01 | 39.4002 | 64.1805 | 0.3499 | 3.3994 | 0.8533 | 3.3823 | 0.7191 | |
| opus-mt2026.01 | 39.3589 | 64.2029 | 0.3523 | 3.3684 | 0.8539 | 3.3569 | 0.7199 | |
| PhiMaestra-32026.01 | 37.8007 | 62.8038 | 0.3153 | 3.2246 | 0.8508 | 3.1435 | 0.7241 | |
| Llama-3.1-8B2026.01 | 36.9176 | 59.6155 | -0.0196 | 5.4246 | 0.8385 | 5.2481 | 0.586 | |
| mBART2026.01 | 35.5482 | 61.1265 | 0.3088 | 3.7333 | 0.8423 | 3.529 | 0.7114 | |
| mBARTbeam_search=5 beams2026.01 | 35.5482 | 61.1265 | 0.3088 | 3.7333 | 0.8423 | 3.529 | 0.7114 | |
| LLaMAntino-8B2026.01 | 34.2716 | 60.8809 | 0.2405 | 3.3702 | 0.8361 | 3.3189 | 0.7117 | |
| Cerbero-7B2026.01 | 33.4609 | 58.6011 | 0.0714 | 4.4285 | 0.8306 | 4.2397 | 0.6516 | |
| ModelloItalia-9B2026.01 | 32.7339 | 61.2928 | 0.0365 | 4.7024 | 0.8133 | 5.4036 | 0.6864 | |
| EuroLLM-9B2026.01 | 32.3876 | 59.65 | 0.2791 | 3.2853 | 0.8453 | 3.1678 | 0.7148 | |
| Maestrale-v0.42026.01 | 30.8264 | 58.1229 | 0.2349 | 3.4905 | 0.8371 | 3.3525 | 0.7036 | |
| EuroLLM-1.7B2026.01 | 27.4118 | 56.2021 | 0.0905 | 4.5263 | 0.8145 | 4.0868 | 0.6674 | |
| Minerva-7B2026.01 | 25.129 | 51.4188 | -0.2554 | 7.1825 | 0.7565 | 6.9447 | 0.5386 |