Machine Translation on WMT En-Ro newstest2016 (test)
35.2BLEUMASS
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MASSArchitecture=6-layer Transformer2019.05 | 35.2 | — | — | |
| ATIterations=N, latency (ms)=4862020.11 | 34.39 | — | — | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=MLM2019.01 | 33.3 | — | — | |
| XLMArchitecture=6-layer Transformer2019.05 | 33.3 | — | — | |
| MASSArchitecture=6-layer Transformer2019.05 | 33.1 | — | — | |
| CMLMIterations=10, latency (ms)=1662020.11 | 33.08 | — | — | |
| LATIterations=4, latency (ms)=732020.11 | 32.87 | — | — | |
| CMLMIterations=4, latency (ms)=722020.11 | 32.53 | — | — | |
| Autoregressivebeam size=42017.11 | 31.91 | 607 | 1 | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=-2019.01 | 31.8 | — | — | |
| XLMArchitecture=6-layer Transformer2019.05 | 31.8 | — | — | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=CLM2019.01 | 31.7 | — | — | |
| XLMEncoder Pre-training=CLM, Decoder Pre-training=MLM2019.01 | 31.6 | — | — | |
| Autoregressivebeam size=12017.11 | 31.35 | 408 | 1.49 | |
| LATIterations=1, latency (ms)=312020.11 | 30.74 | — | — | |
| NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=1002017.11 | 29.79 | 257 | 2.36 | |
| XLMEncoder Pre-training=-, Decoder Pre-training=CLM2019.01 | 29.2 | — | — | |
| NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=102017.11 | 29.02 | 79 | 7.68 | |
| XLMEncoder Pre-training=CLM, Decoder Pre-training=CLM2019.01 | 29 | — | — | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=-2019.01 | 28.2 | — | — | |
| XLMEncoder Pre-training=EMB, Decoder Pre-training=EMB2019.01 | 27.5 | — | — | |
| CMLMIterations=1, latency (ms)=272020.11 | 27.32 | — | — | |
| NATdecoding=argmax, fine-tuning=true2017.11 | 27.29 | 39 | 15.6 | |
| NATdecoding=argmax2017.11 | 26.22 | 39 | 15.6 | |
| XLMEncoder Pre-training=CLM, Decoder Pre-training=-2019.01 | 25.7 | — | — | |
| PBSMT + NMT2019.01 | 25.1 | — | — | |
| PBSMT2019.01 | 21.3 | — | — | |
| NMT2019.01 | 21.2 | — | — | |
| Lample et al. (2018)Architecture=4-layer Transformer2019.05 | 21.18 | — | — | |
| Lample et al. (2018)Architecture=4-layer Transformer2019.05 | 19.44 | — | — | |
| XLMEncoder Pre-training=-, Decoder Pre-training=-2019.01 | 18.9 | — | — |