Machine Translation on WMT 2016 (test)
41.37BLEUCYCLE (REV)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CYCLE (REV)#Params=343M, Training Data=Synthetic2021.04 | 41.37 | — | — | — | — | — | — | — | — | — | |
| CYCLE (REV)#Params=242M, Training Data=Genuine2021.04 | 40.28 | — | — | — | — | — | — | — | — | — | |
| SEQUENCE#Params=242M, Training Data=Genuine2021.04 | 39.93 | — | — | — | — | — | — | — | — | — | |
| CYCLE#Params=242M, Training Data=Genuine2021.04 | 39.82 | — | — | — | — | — | — | — | — | — | |
| Universal#Params=249M, Training Data=Genuine2021.04 | 39.38 | — | — | — | — | — | — | — | — | — | |
| Vanilla#Params=242M, Training Data=Genuine2021.04 | 39.08 | — | — | — | — | — | — | — | — | — | |
| WMT bestParadigm=Supervised2018.09 | 34.2 | — | — | — | — | — | — | — | — | — | |
| Transformer-baseType=Autoregressive, Implementation=Ours2019.09 | 33.09 | — | — | — | — | — | — | — | — | — | |
| CMLM-baseKnowledge Distillation=true, Decoding method=refinement, n=102019.09 | 33.06 | — | — | — | — | — | — | — | — | — | |
| Transformer-baseType=Autoregressive, Implementation=Ours2019.09 | 32.92 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=false, Decoding method=NPD, n=302019.09 | 32.91 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=true, Decoding method=NPD, n=302019.09 | 32.84 | — | — | — | — | — | — | — | — | — | |
| CMLM-baseKnowledge Distillation=false, Decoding method=refinement, n=102019.09 | 32.53 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=true, Decoding method=NPD, n=302019.09 | 32.49 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=false, Decoding method=NPD, n=152019.09 | 32.46 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=false, Decoding method=NPD, n=302019.09 | 32.45 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=true, Decoding method=NPD, n=152019.09 | 32.43 | — | — | — | — | — | — | — | — | — | |
| CMLM-baseKnowledge Distillation=true, Decoding method=refinement, n=102019.09 | 32.42 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=false, Decoding method=NPD, n=302019.09 | 32.35 | — | — | — | — | — | — | — | — | — | |
| CMLM-smallKnowledge Distillation=true, Decoding method=refinement, n=102019.09 | 32.27 | — | — | — | — | — | — | — | — | — | |
| NeoDiffModel type=Hybrid, Beam size=102025.05 | 32.21 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=true, Decoding method=NPD, n=302019.09 | 32.2 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=true, Decoding method=NPD, n=152019.09 | 32.11 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=false, Decoding method=IWD, n=152019.09 | 32.03 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=false, Decoding method=NPD, n=152019.09 | 32.01 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=false, Decoding method=NPD, n=152019.09 | 31.97 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=true, Decoding method=IWD, n=152019.09 | 31.97 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=true, Decoding method=NPD, n=152019.09 | 31.89 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=true, Decoding method=NPD, n=302019.09 | 31.75 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=false, Decoding method=NPD, n=302019.09 | 31.74 | — | — | — | — | — | — | — | — | — | |
| CMLM-smallKnowledge Distillation=true, Decoding method=refinement, n=102019.09 | 31.65 | — | — | — | — | — | — | — | — | — | |
| NeoDiffModel type=Hybrid, Beam size=12025.05 | 31.59 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=true, Decoding method=IWD, n=152019.09 | 31.58 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=false, Decoding method=IWD, n=152019.09 | 31.5 | — | — | — | — | — | — | — | — | — | |
| NAT w/ FTKnowledge Distillation=true, Decoding method=NPD, n=102019.09 | 31.44 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=false, Decoding method=NPD, n=152019.09 | 31.38 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=true, Decoding method=NPD, n=152019.09 | 31.35 | — | — | — | — | — | — | — | — | — | |
| CMLMModel type=Discrete, Beam size=52025.05 | 31.26 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=false, Decoding method=IWD, n=152019.09 | 31.08 | — | — | — | — | — | — | — | — | — | |
| DiNoiSerModel type=Continuous, Beam size=502025.05 | 31.08 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=large, Knowledge Distillation=true, Decoding method=IWD, n=152019.09 | 31.02 | — | — | — | — | — | — | — | — | — | |
| DiNoiSerModel type=Continuous, Beam size=52025.05 | 30.93 | — | — | — | — | — | — | — | — | — | |
| CMLM(MBR)Model type=Discrete, Beam size=52025.05 | 30.92 | — | — | — | — | — | — | — | — | — | |
| CMLM-baseKnowledge Distillation=false, Decoding method=refinement, n=42019.09 | 30.89 | — | — | — | — | — | — | — | — | — | |
| DifformerModel type=Continuous, Beam size=102025.05 | 30.75 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=false, Decoding method=IWD, n=152019.09 | 30.61 | — | — | — | — | — | — | — | — | — | |
| FlowSeqModel size=base, Knowledge Distillation=true, Decoding method=IWD, n=152019.09 | 30.59 | — | — | — | — | — | — | — | — | — | |
| DifformerModel type=Continuous, Beam size=12025.05 | 30.52 | — | — | — | — | — | — | — | — | — | |
| NAT-IRKnowledge Distillation=true, Decoding method=refinement, n=102019.09 | 30.19 | — | — | — | — | — | — | — | — | — | |
| DiffusionLMModel type=Continuous, Beam size=502025.05 | 29.39 | — | — | — | — | — | — | — | — | — | |
| NAT-IRKnowledge Distillation=true, Decoding method=refinement, n=102019.09 | 29.32 | — | — | — | — | — | — | — | — | — | |
| NAT w/ FTKnowledge Distillation=true, Decoding method=NPD, n=102019.09 | 29.02 | — | — | — | — | — | — | — | — | — | |
| DiffusionLMModel type=Continuous, Beam size=52025.05 | 27.01 | — | — | — | — | — | — | — | — | — | |
| SMT (europarl) + w/o lexical reord.Paradigm=Supervised, Lexical Reordering=Excluded2018.09 | 22.2 | — | — | — | — | — | — | — | — | — | |
| SMT (europarl)Paradigm=Supervised2018.09 | 22.12 | — | — | — | — | — | — | — | — | — | |
| SMT (europarl) + constrained vocab.Paradigm=Supervised, Lexical Reordering=Excluded, Vocabulary=Constrained2018.09 | 21.53 | — | — | — | — | — | — | — | — | — | |
| SMT (europarl) + unsup. tuningParadigm=Supervised, Lexical Reordering=Excluded, Vocabulary=Constrained, Tuning=Unsupervised2018.09 | 19.86 | — | — | — | — | — | — | — | — | — | |
| Proposed systemParadigm=Unsupervised2018.09 | 18.23 | — | — | — | — | — | — | — | — | — | |
| Adafactor2025.12 | — | — | 19.015 | — | 23.419 | — | 30.745 | 23.554 | 36.219 | 19.889 | |
| Adam2025.12 | — | — | 18.92 | — | 23.452 | — | 30.831 | 23.634 | 36.246 | 19.831 | |
| Alada2025.12 | — | — | 20.242 | — | 24.113 | — | 30.936 | 24.042 | 36.16 | 20.204 | |
| DirectMode=Fine-tuning2020.10 | — | 14.1 | 19.2 | 21.8 | 26.5 | 20.4 | — | — | — | — | |
| mRASPMode=Fine-tuning2020.10 | — | 17.5 | 22.9 | 24 | 30.9 | 23.8 | — | — | — | — |