Machine Translation on WMT EN-RO 2016
38.3BLEUPaFi
Evaluation Results
| Method | Links | |
|---|---|---|
| PaFi#Tuned=14.2%, #Stored=14.2%2023.05 | 38.3 | |
| PaFi#Tuned=4.5%, #Stored=4.5%2023.05 | 37.7 | |
| MAM Adapter#Tuned=14.2%, #Stored=4.5%2023.05 | 37.5 | |
| Full FT#Tuned=100%, #Stored=100%2023.05 | 37.3 | |
| Pfeiffer Adapter#Tuned=4.8%, #Stored=4.8%2023.05 | 36.9 | |
| HiWi for Weight#Tuned=4.7%, #Stored=4.7%2023.05 | 36.9 | |
| LoRA(ffn)#Tuned=4.1%, #Stored=4.1%2023.05 | 36.8 | |
| Prefix Tuning#Tuned=10.2%, #Stored=2.4%2023.05 | 35.6 | |
| FlowSeq-largeKnowledge Distillation=false, Decoding=argmax2019.09 | 29.86 | |
| FlowSeq-largeKnowledge Distillation=true, Decoding=argmax2019.09 | 29.73 | |
| FlowSeq-baseKnowledge Distillation=true, Decoding=argmax2019.09 | 29.34 | |
| FlowSeq-baseKnowledge Distillation=false, Decoding=argmax2019.09 | 29.26 | |
| Super (11B)Architecture=T5-11B, FLOPs=87e112022.03 | 28.9 | |
| E-LANGArchitecture=T5, Super model=T5-11B, Swift model=T5-Large, Avg. Speed-up (FLOPs)=1.4X, Avg. Speed-up (time)=1.1X2022.03 | 28.9 | |
| Swift (Large)Architecture=T5-Large, FLOPs=4.25e112022.03 | 28.6 | |
| HiWi for Bias#Tuned=4.7%, #Stored=0.02%2023.05 | 28 | |
| NAT w/ FTKnowledge Distillation=true, Decoding=argmax2019.09 | 27.29 | |
| BitFit#Tuned=0.05%, #Stored=0.05%2023.05 | 26.4 | |
| NAT-IRKnowledge Distillation=true, Decoding=argmax2019.09 | 24.45 | |
| CMLM-baseKnowledge Distillation=true, Decoding=argmax2019.09 | 23.65 | |
| Masked Label Smoothing (MLS)Vocabulary Sharing=true2022.03 | 20.88 | |
| TransformerVocabulary Sharing=false, Label Smoothing=true2022.03 | 20.68 | |
| TransformerVocabulary Sharing=true, Label Smoothing=true2022.03 | 20.59 | |
| CMLM-baseKnowledge Distillation=false, Decoding=argmax2019.09 | 20.24 | |
| CMLM-smallKnowledge Distillation=true, Decoding=argmax2019.09 | 20.12 | |
| CTC LossKnowledge Distillation=true, Decoding=argmax2019.09 | 19.93 | |
| TransformerVocabulary Sharing=true, Label Smoothing=false2022.03 | 19.91 | |
| TransformerVocabulary Sharing=false, Label Smoothing=false2022.03 | 19.61 |