Machine Translation on WMT Ro-En '16
38.5BLEU ScoreXLM
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| XLMPretraining=MLM, Training Configuration=ro <-> en + BT2019.01 | 38.5 | — | — | |
| XLMPretraining=CLM, Training Configuration=ro <-> en + BT2019.01 | 37 | — | — | |
| XLMPretraining=MLM, Training Configuration=ro <-> en2019.01 | 35.6 | — | — | |
| XLMPretraining=MLM, Training Configuration=ro -> en2019.01 | 35.3 | — | — | |
| XLMPretraining=None, Training Configuration=ro <-> en + BT2019.01 | 34.4 | — | — | |
| Sennrich et al.Pretraining=None, Note=Uses back-translation and ensemble2019.01 | 33.9 | — | — | |
| TransformerMBR=52026.04 | 33.08 | — | — | |
| ARBeam width (b)=42018.02 | 32.06 | — | — | |
| FastDiSSMBR=20, NFE=202026.04 | 31.99 | — | — | |
| FastDiSSMBR=10, NFE=202026.04 | 31.89 | — | — | |
| FastDiSSMBR=20, NFE=52026.04 | 31.81 | — | — | |
| ARBeam width (b)=12018.02 | 31.55 | — | — | |
| FastDiSSMBR=10, NFE=52026.04 | 31.55 | — | — | |
| XLMPretraining=CLM, Training Configuration=ro -> en2019.01 | 31.5 | — | — | |
| XLMPretraining=CLM, Training Configuration=ro <-> en2019.01 | 31.5 | — | — | |
| NATFertility (FT)=true, NPD reranking=100 samples2018.02 | 31.44 | — | — | |
| DINOISERMBR=50, NFE=202026.04 | 31.22 | — | — | |
| CMLMMBR=52026.04 | 31.13 | — | — | |
| FlowSeq-largeKnowledge Distillation=true, Decoding=argmax2019.09 | 30.72 | — | — | |
| FlowSeq-largeKnowledge Distillation=false, Decoding=argmax2019.09 | 30.69 | — | — | |
| FlowSeq-baseKnowledge Distillation=true, Decoding=argmax2019.09 | 30.44 | — | — | |
| Iterative RefinementAdaptive refinement steps=true2018.02 | 30.3 | — | — | |
| Iterative RefinementRefinement steps (i_dec)=102018.02 | 30.19 | — | — | |
| DiffusionLM‡MBR=50, NFE=202026.04 | 30.17 | — | — | |
| FlowSeq-baseKnowledge Distillation=false, Decoding=argmax2019.09 | 30.16 | — | — | |
| Iterative RefinementRefinement steps (i_dec)=52018.02 | 29.72 | — | — | |
| Difformer‡MBR=20, NFE=202026.04 | 29.37 | — | — | |
| NATFertility (FT)=true2018.02 | 29.06 | — | — | |
| NAT w/ FTKnowledge Distillation=true, Decoding=argmax2019.09 | 29.06 | — | — | |
| XLMPretraining=None, Training Configuration=ro <-> en2019.01 | 28.5 | — | — | |
| XLMPretraining=None, Training Configuration=ro -> en2019.01 | 28.4 | — | — | |
| Iterative RefinementRefinement steps (i_dec)=22018.02 | 28.15 | — | — | |
| Iterative RefinementRefinement steps (i_dec)=12018.02 | 25.73 | — | — | |
| NAT-IRKnowledge Distillation=true, Decoding=argmax2019.09 | 25.73 | — | — | |
| CTC LossKnowledge Distillation=true, Decoding=argmax2019.09 | 24.71 | — | — | |
| CMLM-baseKnowledge Distillation=true, Decoding=argmax2019.09 | 22.78 | — | — | |
| CMLM-smallKnowledge Distillation=true, Decoding=argmax2019.09 | 20.36 | — | — |