WMT
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
WMT22 (de, cs, zh, ru) and WMT21 (is) (test)
67.88COMET (de)
4
WMT en -> xx 24++
87.6Pass@1
4
WMT16 Instruct
35.564Linewise BLEU
4
WMT English-Czech 19
21.5BLEU
4
WMT'14 De-En (newstest2014)
34.94BLEU
4
WMT lt→en 2019 (val)
17.54BLEU
4
WMT cs→en 2019 (val)
27.63BLEU
4
WMT 23 (test)
42.8BLEU
4
WMT En-De ’24 (test)
97.1COMET
4
WMT En-De 23 (dev)
97.3COMET Score
4
WMT En-Ja ’24 (test)
91.1COMET
4
WMT En-Ja ’23 (dev)
93.2COMET
4
WMT en-ru 24
0.625Pearson R
4
WMT en-cn 24
0.537Pearson Correlation
4
WMT en-ko 24
0.626Pearson Correlation
4
WMT en-it 24
0.588Pearson R
4
WMT en-nl 24
0.552Pearson Correlation
4
WMT en-pt 24
0.533Pearson Correlation
4
WMT en-fr 24
0.53Pearson Correlation
4
WMT 22
0.838MT Score (DE->EN)
4
WMT En-Ja 2024 (test)
0.81XCOMET-XL
4
WMT En-De 2024 (test)
85.5XCOMET-XL
4
WMT Finnish-to-English 2016 (test)
55.8BLEU-1
4
WMT Czech-to-English 2016 (test)
0.611BLEU-1
4
WMT22 Liv-En (test)
43.6COMET Score
4