Machine Translation on EC-40
58.84chrF++ (English, Fwd)MITRE
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MITRENumber of Parameters=913M2025.01 | 58.84 | 64.01 | 52.4 | 54.59 | 51.03 | 55.36 | 50.32 | 49.84 | 48.97 | 52.88 | 46.65 | 33.88 | 50.42 | |
| MITRENumber of Parameters=466M2025.01 | 58.11 | 62.77 | 51.4 | 53.41 | 50.06 | 54.46 | 49.18 | 48.62 | 47.83 | 52.04 | 45.1 | 32.41 | 49.29 | |
| GPTModel Version=4o mini2025.01 | 58.03 | 62.85 | 51.26 | 52.9 | 49.33 | 54.33 | 49.12 | 48.62 | 49.39 | 52.25 | 45.86 | 34.11 | 49.48 | |
| NLLBNumber of Parameters=3.3B2025.01 | 57.88 | 64.27 | 50.95 | 53.16 | 49.63 | 53.92 | 48.91 | 48.76 | 49.06 | 53.28 | 45.71 | 31.97 | 49.35 | |
| NLLBNumber of Parameters=1.3B2025.01 | 56.82 | 63.35 | 50.07 | 52.21 | 48.8 | 53.19 | 47.98 | 47.46 | 47.92 | 52.49 | 44.7 | 30.98 | 48.4 | |
| NLLBNumber of Parameters=615M2025.01 | 55.54 | 61.73 | 48.48 | 50.39 | 47.38 | 51.54 | 46.38 | 45.34 | 45.98 | 50.96 | 42.56 | 29.73 | 46.7 | |
| GPTModel Version=3.5 turbo2025.01 | 55.3 | 61.6 | 49.39 | 52.16 | 48.31 | 53.34 | 47.54 | 46.35 | 45.64 | 48.05 | 43.46 | 31.2 | 47.41 | |
| M2MNumber of Parameters=1.2B2025.01 | 54.8 | 54.44 | 49.02 | 47.06 | 47.49 | 48.04 | 45.87 | 43.35 | 32.78 | 40.68 | 30.9 | 28.01 | 42.56 | |
| M2MNumber of Parameters=483M2025.01 | 50.43 | 54.36 | 44.84 | 46.24 | 43.96 | 48.26 | 43.36 | 43.06 | 37.54 | — | 39.83 | 27.85 | 42.53 | |
| M2MNumber of Parameters=615M2025.01 | 49.97 | 54.74 | 46.77 | 48.62 | 45.34 | 49.83 | 44.7 | 44.42 | 40.17 | 43.59 | 41.04 | 28.84 | 44.12 |