Machine Translation on FLORES-200 (it-en, devtest)
37.0319sacreBLEUGemmaX2-9B
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GemmaX2-9B2026.01 | 37.0319 | 64.5113 | 0.4534 | 2.0697 | 0.886 | 2.1575 | 0.7909 | |
| GemmaX2-9B-b5Beam search=5 beams2026.01 | 36.5996 | 64.562 | 0.4526 | 2.0466 | 0.886 | 2.1405 | 0.793 | |
| Tower-7B2026.01 | 35.6008 | 63.6222 | 0.4422 | 2.1038 | 0.8841 | 2.1634 | 0.7911 | |
| GemmaX2-2B2026.01 | 35.2042 | 63.5271 | 0.4335 | 2.1454 | 0.8828 | 2.1597 | 0.7908 | |
| Madlad-7B2026.01 | 35.0758 | 63.5949 | 0.4384 | 2.1343 | 0.8833 | 2.1693 | 0.7916 | |
| GemmaX2-2B-b5Beam search=5 beams2026.01 | 35.0271 | 63.6901 | 0.4354 | 2.1187 | 0.8826 | 2.1311 | 0.7926 | |
| NLLB-1.3B2026.01 | 34.9951 | 62.3869 | 0.4182 | 2.4079 | 0.8772 | 2.4782 | 0.7797 | |
| NLLB-1.3B-b5Beam search=5 beams2026.01 | 34.8996 | 62.8399 | 0.4277 | 2.2598 | 0.8791 | 2.3067 | 0.7856 | |
| NLLB-3.3B-b5Beam search=5 beams2026.01 | 34.865 | 63.4208 | 0.4363 | 2.1519 | 0.8821 | 2.1906 | 0.7892 | |
| NLLB-3.3B2026.01 | 34.4729 | 62.9651 | 0.4271 | 2.258 | 0.8798 | 2.3201 | 0.7849 | |
| Madlad-7B-b5Beam search=5 beams2026.01 | 34.3254 | 63.5181 | 0.4323 | 2.1115 | 0.8821 | 2.1572 | 0.7921 | |
| Madlad-3B2026.01 | 34.2333 | 63.0402 | 0.4295 | 2.1811 | 0.8814 | 2.1863 | 0.7905 | |
| Madlad-3B-b5Beam search=5 beams2026.01 | 34.0046 | 63.048 | 0.4282 | 2.1624 | 0.8807 | 2.1629 | 0.7917 | |
| NLLB-600M-b5Beam search=5 beams2026.01 | 33.4228 | 61.6058 | 0.3999 | 2.4997 | 0.8722 | 2.4201 | 0.78 | |
| DIETA+allsynth2026.01 | 33.3923 | 62.0162 | 0.4035 | 2.4848 | 0.875 | 2.4362 | 0.7787 | |
| NLLB-600M2026.01 | 33.3599 | 60.8455 | 0.3869 | 2.6995 | 0.8681 | 2.64 | 0.7708 | |
| DIETA+cont2026.01 | 33.1475 | 62.0151 | 0.4012 | 2.4644 | 0.8749 | 2.3798 | 0.7817 | |
| OpusMT-Big2026.01 | 32.8311 | 62.1205 | 0.3985 | 2.4917 | 0.8754 | 2.3902 | 0.7839 | |
| OpusMT-Big-b5Beam search=5 beams2026.01 | 32.8119 | 62.1055 | 0.3988 | 2.4878 | 0.8753 | 2.3851 | 0.7776 | |
| DIETA+nosynth2026.01 | 32.5469 | 61.6133 | 0.395 | 2.4962 | 0.874 | 2.3991 | 0.7796 | |
| DIETA2026.01 | 32.108 | 61.2657 | 0.3956 | 2.5852 | 0.8729 | 2.4924 | 0.7777 | |
| PhiMaestra-32026.01 | 31.1726 | 60.626 | 0.3851 | 2.4697 | 0.8722 | 2.3772 | 0.7791 | |
| DIETA+BT2026.01 | 30.3901 | 60.276 | 0.3662 | 2.9009 | 0.8662 | 2.7266 | 0.766 | |
| OpusMT2026.01 | 29.3934 | 59.9987 | 0.3542 | 2.7883 | 0.8656 | 2.5676 | 0.7784 | |
| OpusMT-b5Beam search=5 beams2026.01 | 29.382 | 59.9722 | 0.3545 | 2.785 | 0.8658 | 2.5677 | 0.7785 | |
| Cerbero-7B2026.01 | 29.1301 | 58.8351 | 0.2666 | 3.0239 | 0.8653 | 2.8194 | 0.7422 | |
| LLaMAntino-8B2026.01 | 28.3368 | 59.5429 | 0.3518 | 2.6661 | 0.8661 | 2.5392 | 0.7723 | |
| EuroLLM-9B2026.01 | 27.9216 | 58.8887 | 0.3767 | 2.4993 | 0.8699 | 2.374 | 0.7736 | |
| Maestrale-v0.42026.01 | 27.7433 | 58.3707 | 0.3407 | 2.5526 | 0.8674 | 2.474 | 0.7619 | |
| mBART502026.01 | 27.3513 | 57.6473 | 0.2913 | 3.5905 | 0.8494 | 3.0786 | 0.7439 | |
| mBART50-b5Beam search=5 beams2026.01 | 27.3513 | 57.6473 | 0.2913 | 3.5905 | 0.8494 | 3.0786 | 0.7439 | |
| Llama3.1-8B-ITA2026.01 | 27.2647 | 54.862 | 0.0412 | 4.725 | 0.8573 | 4.3978 | 0.6258 | |
| ModItalia-9B2026.01 | 26.0021 | 57.9634 | 0.0578 | 4.0703 | 0.829 | 4.4706 | 0.7369 | |
| EuroLLM-1.7B2026.01 | 23.3948 | 55.4949 | 0.1863 | 3.5958 | 0.8415 | 3.1847 | 0.7321 | |
| Minerva-7B2026.01 | 23.0404 | 52.7627 | -0.106 | 6.2368 | 0.794 | 6.0661 | 0.6136 |