Machine Translation on FLORES (test)
64.4chrF2 ScoreMixed-Soup
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mixed-SoupLanguage=tam2026.06 | 64.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=tam2026.06 | 64.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=tel2026.06 | 62.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=tel2026.06 | 62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=mal2026.06 | 61.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=mal2026.06 | 61.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=hin2026.06 | 60 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=hin2026.06 | 59.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=npi2026.06 | 59.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=kan2026.06 | 59 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=npi2026.06 | 59 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=kan2026.06 | 58.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=ben2026.06 | 58.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=ben2026.06 | 58.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=mai2026.06 | 56.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=mai2026.06 | 56.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=guj2026.06 | 55.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=pan2026.06 | 55.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=mar2026.06 | 55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=pan2026.06 | 54.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=guj2026.06 | 54.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=urd2026.06 | 54.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=urd2026.06 | 54.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=mar2026.06 | 54.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=ory2026.06 | 52 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=ory2026.06 | 51.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=doi2026.06 | 49.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=doi2026.06 | 49.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=gom2026.06 | 49.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=gom2026.06 | 49.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=mni2026.06 | 48.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=mni2026.06 | 48.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=brx2026.06 | 46 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=brx2026.06 | 45.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=asm2026.06 | 44.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=asm2026.06 | 44.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=kas2026.06 | 39.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=kas2026.06 | 38.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=san2026.06 | 36.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=san2026.06 | 36.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BaseLanguage=sat2026.06 | 30.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Mixed-SoupLanguage=sat2026.06 | 30.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Bielik-11B-v2Params (B)=112026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.25 | 14.86 | 7.64 | |
| Bielik-11B-v2.6-InstructParams (B)=112026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 13.58 | 15.77 | 11.38 | |
| Bielik-11B-v3.0Params (B)=112026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 17.85 | 17.6 | 18.11 | |
| Bielik-11B-v3.0-InstructParams (B)=112026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 19.22 | 18.54 | 19.91 | |
| Bielik-Minitron-7B-v3.0-InstructParams (B)=72026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 15.53 | 15.74 | 15.32 | |
| chain-gloss2024.10 | — | 44.37 | 24.56 | 43.47 | 44.62 | 26.12 | 45.82 | 48.97 | 54.86 | 65.15 | 47.77 | 57.29 | 55.42 | 46.54 | — | — | — | |
| EuroLLM-9B-Instruct*Params (B)=92026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 20.61 | 19.28 | 21.95 | |
| few-shot2024.10 | — | 45.83 | 23.99 | 43.93 | 47.06 | 26.1 | 47.93 | 50.67 | 55.14 | 65.75 | 47.1 | 55.15 | 57.05 | 47.14 | — | — | — | |
| gloss-shot2024.10 | — | 47.36 | 25.53 | 45 | 46.67 | 25.31 | 47.4 | 50.19 | 57.24 | 67.21 | 50.2 | 58.32 | 57.04 | 48.12 | — | — | — | |
| Mistral-Nemo-Instruct-2407Params (B)=122026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.35 | 13.37 | 15.33 | |
| model-gloss2024.10 | — | 43 | 21.79 | 41.17 | 45.11 | 23.1 | 46.3 | 49.5 | 56.62 | 67.33 | 49.85 | 58.32 | 56.54 | 46.55 | — | — | — | |
| phi-4 (15B)Params (B)=152026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 15.58 | 14.55 | 16.61 | |
| Qwen2.5-14B-InstructParams (B)=142026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 13.24 | 12.55 | 13.93 | |
| Qwen2.5-7B-InstructParams (B)=72026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.34 | 10.43 | 12.26 | |
| Qwen3-14BParams (B)=142026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 15.37 | 14.18 | 16.56 |