Machine Translation on WMT En-Fr 2014
43.8BLEUAdmin
Evaluation Results
| Method | Links | |
|---|---|---|
| AdminParam. #=262 M, dim(W(1)) in FFN=512 × 2048, Enc#-Dec#=60L-12L2020.04 | 43.8 | |
| PrimeParam. #=252 M, dim(W(1)) in FFN=1024 × 4096, Enc#-Dec#=6L-6L2020.04 | 43.48 | |
| T5-11BParam. #=11 B, dim(W(1)) in FFN=1024 × 65536, Enc#-Dec#=24L-24L2020.04 | 43.4 | |
| DG-TransformerParam. #=264 M, dim(W(1)) in FFN=1024 × 4096, Enc#-Dec#=8L-8L2020.04 | 43.27 | |
| DynamicConvParam. #=213 M, dim(W(1)) in FFN=1024 × 4096, Enc#-Dec#=7L-7L2020.04 | 43.2 | |
| Pre-LNParam. #=262 M, dim(W(1)) in FFN=512 × 2048, Enc#-Dec#=60L-12L2020.04 | 43.1 | |
| T5-3BParam. #=3 B, dim(W(1)) in FFN=1024 × 16384, Enc#-Dec#=24L-24L2020.04 | 42.6 | |
| T5-LargeParam. #=770 M, dim(W(1)) in FFN=1024 × 4096, Enc#-Dec#=12L-12L2020.04 | 41.5 | |
| T5-BaseParam. #=220 M, dim(W(1)) in FFN=512 × 2048, Enc#-Dec#=6L-6L2020.04 | 41.2 | |
| Trans.Big-RNMT+Param. #=377 M, dim(W(1)) in FFN=1024 × 8192, Enc#-Dec#=6L-6L2020.04 | 41.12 | |
| XLMApproach=+Self-training2022.03 | 39.3 | |
| MASSApproach=UNMT2022.03 | 39.2 | |
| XLMApproach=UNMT2022.03 | 39.1 | |
| MASSApproach=+Self-training2022.03 | 39 | |
| CBDImplementation=Existing Works2022.03 | 38.2 | |
| MASSImplementation=Our Implementation, Approach=+Self-training2022.03 | 38 | |
| XLMImplementation=Our Implementation, Approach=+Self-training2022.03 | 37.8 | |
| MASSImplementation=Our Implementation, Approach=UNMT2022.03 | 37.8 | |
| MASSImplementation=Existing Works, Approach=UNMT2022.03 | 37.5 | |
| XLMImplementation=Our Implementation, Approach=UNMT2022.03 | 37.4 | |
| CBDImplementation=Existing Works2022.03 | 35.5 | |
| MASSImplementation=Our Implementation, Approach=+Self-training2022.03 | 35.2 | |
| XLMImplementation=Our Implementation, Approach=+Self-training2022.03 | 35.1 | |
| Supervised Modelmode=supervised2021.09 | 35 | |
| MASSImplementation=Existing Works, Approach=UNMT2022.03 | 34.9 | |
| MASSImplementation=Our Implementation, Approach=UNMT2022.03 | 34.9 | |
| XLMImplementation=Our Implementation, Approach=UNMT2022.03 | 34.5 | |
| FLAN 137Bprotocol=zero-shot, template_selection=best dev, number_of_templates=52021.09 | 33.9 | |
| FLAN 137Bprotocol=few-shot, template_selection=average, k=9, number_of_templates=52021.09 | 33.9 | |
| FLAN 137Bprotocol=few-shot, template_selection=best dev, k=9, number_of_templates=52021.09 | 33.8 | |
| XLMImplementation=Existing Works, Approach=UNMT2022.03 | 33.4 | |
| XLMImplementation=Existing Works, Approach=UNMT2022.03 | 33.3 | |
| FLAN 137Bprotocol=zero-shot, template_selection=average, number_of_templates=52021.09 | 32.9 | |
| GPT-3 175Bprotocol=few-shot, k=642021.09 | 32.6 | |
| LaMDA-PTprotocol=few-shot, k=52021.09 | 31.5 | |
| GPT-3 175Bprotocol=zero-shot2021.09 | 25.2 | |
| LaMDA-PTprotocol=zero-shot2021.09 | 11.2 | |
| TransEvolve-fullFF-2#Params=59M2021.09 | 0.395 | |
| Transformer BASE#Params=65M2021.09 | 0.381 | |
| TransEvolve-fullFF-1#Params=53M2021.09 | 0.38 | |
| TransEvolve-randomFF-2#Params=33M2021.09 | 0.334 | |
| TransEvolve-randomFF-1#Params=27M2021.09 | 0.326 |