Machine Translation on WMT16 English-German (test)
41.2BLEUSOTA (Supervised)
Evaluation Results
| Method | Links | |
|---|---|---|
| SOTA (Supervised)Training=Supervised2020.05 | 41.2 | |
| Finetuned SOTAprotocol=Supervised2022.04 | 41.2 | |
| PaLM 540Bshots=52022.04 | 37.4 | |
| SEQUENCEM=6, N=12, #Params=61M, Speed=x1.312021.04 | 34.99 | |
| CYCLE (REV)M=6, N=18, #Params=61M, Speed=x0.982021.04 | 34.84 | |
| CYCLE (REV)M=6, N=12, #Params=61M, Speed=x1.312021.04 | 34.81 | |
| SEQUENCEM=6, N=18, #Params=61M, Speed=x0.982021.04 | 34.8 | |
| CYCLEM=6, N=18, #Params=61M, Speed=x0.982021.04 | 34.79 | |
| CYCLEM=6, N=12, #Params=61M, Speed=x1.312021.04 | 34.77 | |
| UniversalM=1, N=6, #Params=63M, Speed=x1.002021.04 | 34.6 | |
| VanillaM=6, N=6, #Params=61M, Speed=x2.022021.04 | 34.23 | |
| Vanilla (deep)M=18, N=18, #Params=149M, Speed=x0.962021.04 | 34.19 | |
| Universal (deep)M=1, N=12, #Params=63M, Speed=x0.522021.04 | 34.01 | |
| Vanilla (big)M=6, N=6, #Params=210M, Speed=x0.812021.04 | 33.4 | |
| Universal (small)M=1, N=6, #Params=24M, Speed=x2.522021.04 | 32.81 | |
| Supervised NMTUSMT Tuning=Supervised, Sentence pairs=5.6M2018.10 | 32.3 | |
| PaLM 540Bshots=02022.04 | 31.8 | |
| PaLM 540Bshots=12022.04 | 31.8 | |
| Supervised NMTUSMT Tuning=Supervised, Sentence pairs=2.8M2018.10 | 31.6 | |
| Supervised NMTUSMT Tuning=Supervised, Sentence pairs=1.4M2018.10 | 29.9 | |
| mBARTTraining=Unsupervised2020.05 | 29.8 | |
| GPT-3Few-shot learning=Few-shot2020.05 | 29.7 | |
| MASSTraining=Unsupervised2020.05 | 28.3 | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=-2019.01 | 27 | |
| FLANshots=112022.04 | 26.8 | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=MLM2019.01 | 26.4 | |
| XLMTraining=Unsupervised2020.05 | 26.4 | |
| GPT-3Few-shot learning=One-shot2020.05 | 26.2 | |
| GPT-3 175Bshots=12022.04 | 26.2 | |
| FLANshots=02022.04 | 25.4 | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=CLM2019.01 | 24.9 | |
| GPT-3Few-shot learning=Zero-shot2020.05 | 24.6 | |
| XLMEncoder Pre-training=-, Decoder Pre-training=CLM2019.01 | 24.4 | |
| Supervised trainingTraining mode=Supervised, Loss=Standard cross-entropy2018.04 | 24.07 | |
| XLMEncoder Pre-training=CLM, Decoder Pre-training=CLM2019.01 | 22.7 | |
| UNMT (this work)USMT Tuning=Supervised, Filtering=Enabled2018.10 | 21.6 | |
| XLMEncoder Pre-training=MLM, Decoder Pre-training=-2019.01 | 21.6 | |
| UNMT (this work) w/o filteringUSMT Tuning=Supervised2018.10 | 21.3 | |
| XLMEncoder Pre-training=EMB, Decoder Pre-training=EMB2019.01 | 21.3 | |
| XLMEncoder Pre-training=CLM, Decoder Pre-training=MLM2019.01 | 21.3 | |
| Lample et al. (2018b) USMT+UNMTUSMT Tuning=No2018.10 | 20.2 | |
| PBSMT + NMT2019.01 | 20.2 | |
| UNMT (this work)USMT Tuning=No, Filtering=Enabled2018.10 | 20 | |
| UNMT (this work) w/o filteringUSMT Tuning=No2018.10 | 19.6 | |
| XLMEncoder Pre-training=CLM, Decoder Pre-training=-2019.01 | 19.2 | |
| Artetxe et al. (2018b) USMTUSMT Tuning=back-translation2018.10 | 18.2 | |
| Lample et al. (2018b) USMTUSMT Tuning=No2018.10 | 17.8 | |
| PBSMT2019.01 | 17.8 | |
| USMT (this work) w/ forward translationUSMT Tuning=Supervised2018.10 | 17.4 | |
| Lample et al. (2018b) UNMTUSMT Tuning=No2018.10 | 17.2 | |
| NMT2019.01 | 17.2 | |
| USMT (this work) w/ back-translationUSMT Tuning=Supervised2018.10 | 17 | |
| USMT (this work) w/ forward translationUSMT Tuning=No2018.10 | 15.5 | |
| USMT (this work) w/ back-translationUSMT Tuning=No2018.10 | 15 | |
| Unsupervised Neural Machine Translation with Weight SharingWeight-sharing layers=12018.04 | 10.86 | |
| Lample et al.2018.04 | 9.64 | |
| XLMEncoder Pre-training=-, Decoder Pre-training=-2019.01 | 6.7 | |
| Word-by-wordMethod type=Baseline2018.04 | 5.85 |