Machine Translation on WMT En-De 2022 (test)
87.4COMETWMT22 Winners
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| WMT22 Winners2024.06 | 87.4 | — | — | 38.4 | |
| GPT-3.5-turboprompt=SENSE2024.09 | 86.44 | — | 59.08 | 33.75 | |
| NLLB-3.3b2024.06 | 86.23 | — | — | 33.98 | |
| GPT-3.5-turboprompt=Vanilla2024.09 | 85.71 | — | 58.19 | 33.15 | |
| BayLing2024.09 | 85.47 | — | 58.65 | 32.94 | |
| DTGshot=5-shot2024.09 | 85.4 | — | 58.2 | 33.2 | |
| WMT-Best2024.09 | 85 | — | 58.5 | 33.4 | |
| GPT-3.5-turboprompt=COT2024.09 | 84.99 | — | 57.74 | 31.46 | |
| GPT EVAL2024.09 | 84.8 | — | 58.3 | 33.4 | |
| TASTEBackbone=LLaMA-2-7b, Training Strategy=Tuning with Fixed Embedding Layer, Subtask Form=Text Classification2024.06 | 83.8 | — | — | 27.94 | |
| MT-FullBackbone=LLaMA-2-7b, Training Strategy=Full-Parameter Tuning2024.06 | 83.7 | — | — | 27.18 | |
| TASTEBackbone=LLaMA-2-7b, Training Strategy=Tuning with Fixed Embedding Layer, Subtask Form=Quality Estimation2024.06 | 83.7 | — | — | 27.32 | |
| MT-FixEmbBackbone=LLaMA-2-7b, Training Strategy=Tuning with Fixed Embedding Layer2024.06 | 83.66 | — | — | 27.75 | |
| TASTEBackbone=LLaMA-2-7b, Training Strategy=Full-Parameter Tuning, Subtask Form=Quality Estimation2024.06 | 83.38 | — | — | 27.16 | |
| TASTEBackbone=LLaMA-2-7b, Training Strategy=Full-Parameter Tuning, Subtask Form=Text Classification2024.06 | 82.95 | — | — | 26.74 | |
| BaylingBackbone=LLaMA-2-7b2024.06 | 82.18 | — | — | 25.66 | |
| ParroTBackbone=LLaMA-2-7b2024.06 | 81.6 | — | — | 26.1 | |
| TASTEBackbone=BLOOMZ-7b1-mt, Training Strategy=Tuning with Fixed Embedding Layer, Subtask Form=Text Classification2024.06 | 76.38 | — | — | 21.98 | |
| TASTEBackbone=BLOOMZ-7b1-mt, Training Strategy=Tuning with Fixed Embedding Layer, Subtask Form=Quality Estimation2024.06 | 76.3 | — | — | 21.84 | |
| TASTEBackbone=BLOOMZ-7b1-mt, Training Strategy=Full-Parameter Tuning, Subtask Form=Text Classification2024.06 | 75.53 | — | — | 21.13 | |
| TASTEBackbone=BLOOMZ-7b1-mt, Training Strategy=Full-Parameter Tuning, Subtask Form=Quality Estimation2024.06 | 75.41 | — | — | 21.18 | |
| TIMBackbone=BLOOMZ-7b1-mt2024.06 | 74.91 | — | — | 20.9 | |
| ParroTBackbone=BLOOMZ-7b1-mt2024.06 | 73.6 | — | — | 20.5 | |
| MT-FixEmbBackbone=BLOOMZ-7b1-mt, Training Strategy=Tuning with Fixed Embedding Layer2024.06 | 72.06 | — | — | 19.41 | |
| MT-FullBackbone=BLOOMZ-7b1-mt, Training Strategy=Full-Parameter Tuning2024.06 | 71.31 | — | — | 18.84 | |
| Baseline2022.09 | — | 26.4 | — | — | |
| BayLingScale=7B2023.11 | — | 28.2 | 53.6 | — | |
| BayLingScale=7B2023.11 | — | 25.7 | 53.6 | — | |
| Best Officialsystem setting=constrained2022.09 | — | 37.8 | — | — | |
| GoogleMT2023.11 | — | 34.1 | 59.1 | — | |
| GoogleMT2023.11 | — | 37.5 | 64.7 | — | |
| MicroMT2023.11 | — | 33.9 | 58.8 | — | |
| MicroMT2023.11 | — | 37.5 | 64.7 | — | |
| TIMScale=13B2023.11 | — | 31.7 | 56.9 | — | |
| TIMScale=13B2023.11 | — | 33.2 | 60.8 | — | |
| Vega-MT2022.09 | — | 37.8 | — | — | |
| WMT-Best2023.11 | — | 33.4 | 58.5 | — | |
| WMT-Best2023.11 | — | 38.4 | 64.6 | — |