Machine Translation on Flores (dev)
41.49spBLEU (En ->)GPT
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPTVersion=4o mini2025.01 | 41.49 | 43.97 | 33.09 | 31.92 | 31.4 | 34.03 | 30.54 | 30.69 | 31.01 | 27.2 | 26.34 | 27.53 | 31.09 | |
| MITREParameters=913M2025.01 | 41.16 | 44.17 | 33.34 | 32.95 | 32.53 | 34.23 | 30.74 | 31.26 | 29.9 | 27.22 | 25.93 | 25.58 | 31.15 | |
| MITREParameters=466M2025.01 | 40.2 | 42.6 | 32.14 | 31.51 | 31.32 | 33.26 | 29.36 | 29.8 | 28.46 | 26.16 | 24.05 | 23.56 | 29.77 | |
| NLLBParameters=3.3B2025.01 | 39.8 | 45.08 | 31.93 | 31.77 | 30.88 | 32.62 | 29.29 | 30.13 | 29.81 | 28.08 | 25.18 | 22.56 | 30.01 | |
| GPTVersion=3.5 turbo2025.01 | 38.27 | 42.37 | 31.01 | 31.02 | 30.09 | 32.73 | 28.56 | 27.85 | 26.75 | 22.81 | 23.61 | 24.08 | 28.66 | |
| NLLBParameters=1.3B2025.01 | 38.08 | 43.42 | 30.52 | 30.17 | 29.63 | 31.42 | 27.84 | 28.25 | 28.08 | 26.87 | 23.5 | 21.06 | 28.51 | |
| M2MParameters=1.2B2025.01 | 35.92 | 35.14 | 29.51 | 28.4 | 28.49 | 28.38 | 26.58 | 26.19 | 18.09 | 17.57 | 15.48 | 20.07 | 24.69 | |
| NLLBParameters=615M2025.01 | 35.85 | 41.04 | 28.13 | 27.41 | 27.46 | 29.09 | 25.4 | 25.33 | 25.39 | 24.35 | 20.72 | 19.42 | 26.05 | |
| M2MParameters=615M2025.01 | 30.69 | 31.98 | 26.35 | 25.56 | 26.82 | 27.52 | 24.02 | 24.77 | 20.11 | 17.49 | 19.31 | 19.09 | 23.65 | |
| M2MParameters=483M2025.01 | 30.36 | 31.92 | 24.4 | 22.58 | 24.01 | 25.81 | 22.59 | 23.4 | 17.94 | 16.5 | 18.3 | 18.37 | 22.1 |