Machine Translation on EC-40 (test)
87.5COMET (Germanic)GPT 4o mini
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT 4o mini2025.01 | 87.5 | 87.58 | 87.08 | 85.89 | 86.99 | 87.16 | — | |
| MITRE#params=913M2025.01 | 86.54 | 86.96 | 86.16 | 83.15 | 85.52 | 85.88 | — | |
| NLLB#params=3.3B2025.01 | 86.49 | 86.58 | 85.84 | 84.63 | 85.42 | 85.96 | — | |
| GPT 4o mini2025.01 | 86.38 | 86.57 | 88.9 | 86.03 | 87.47 | 87.16 | 89.59 | |
| GPT 3.5 turbo2025.01 | 86.26 | 86.62 | 85.91 | 83.09 | 85.45 | 85.66 | — | |
| MITRE#params=466M2025.01 | 85.99 | 86.49 | 85.58 | 82.24 | 84.38 | 85.19 | — | |
| GPT 3.5 turbo2025.01 | 85.8 | 85.96 | 87.5 | 82.09 | 85.77 | 85.66 | 87.67 | |
| NLLB#params=1.3B2025.01 | 85.72 | 85.93 | 85.07 | 83.57 | 84.36 | 85.13 | — | |
| MITRE#params=913M2025.01 | 85.61 | 85.7 | 88.03 | 83.8 | 85.35 | 85.88 | 88.11 | |
| NLLB#params=3.3B2025.01 | 85.58 | 85.45 | 87.96 | 85.45 | 84.56 | 85.96 | 88.22 | |
| MITRE#params=466M2025.01 | 84.98 | 85.14 | 87.19 | 83.41 | 84.29 | 85.19 | 87.87 | |
| NLLB#params=1.3B2025.01 | 84.76 | 84.76 | 86.82 | 84.93 | 83.52 | 85.13 | 87.76 | |
| M2M#params=1.2B2025.01 | 84.15 | 84.46 | 83.87 | 75.91 | 82.95 | 82.66 | — | |
| NLLB#params=615M2025.01 | 84.06 | 84.5 | 83.41 | 81.26 | 82.12 | 83.33 | — | |
| M2M#params=1.2B2025.01 | 82.87 | 83.12 | 85.41 | 77.83 | 82.58 | 82.66 | 85.93 | |
| NLLB#params=615M2025.01 | 82.77 | 83.05 | 84.39 | 83.51 | 82.02 | 83.33 | 86.61 | |
| M2M#params=615M2025.01 | 81.42 | 82 | 81.29 | 72.56 | 80.02 | 79.79 | — | |
| M2M#params=615M2025.01 | 79.98 | 80.5 | 82.43 | 74.62 | 79.96 | 79.79 | 81.16 | |
| M2M#params=483M2025.01 | 78.9 | 80.48 | 79.85 | 68.34 | 78.67 | 77.74 | — | |
| M2M#params=483M2025.01 | 77.03 | 78.49 | 80.31 | 72.75 | 78.57 | 77.74 | 81.63 |