List your company
ResearchFormerly WizWand

Machine Translation on WMT EN-RO 2016

38.3BLEUPaFi
18.862423.908728.95534.0013Sep 5, 2019Apr 18, 2020Dec 1, 2020Jul 15, 2021Feb 27, 2022Oct 11, 2022May 26, 2023
Updated 8mo ago

Evaluation results

MethodLinks
PaFi#Tuned=14.2%, #Stored=14.2%2023.0538.3
PaFi#Tuned=4.5%, #Stored=4.5%2023.0537.7
MAM Adapter#Tuned=14.2%, #Stored=4.5%2023.0537.5
Full FT#Tuned=100%, #Stored=100%2023.0537.3
Pfeiffer Adapter#Tuned=4.8%, #Stored=4.8%2023.0536.9
HiWi for Weight#Tuned=4.7%, #Stored=4.7%2023.0536.9
LoRA(ffn)#Tuned=4.1%, #Stored=4.1%2023.0536.8
Prefix Tuning#Tuned=10.2%, #Stored=2.4%2023.0535.6
FlowSeq-largeKnowledge Distillation=false, Decoding=argmax2019.0929.86
FlowSeq-largeKnowledge Distillation=true, Decoding=argmax2019.0929.73
FlowSeq-baseKnowledge Distillation=true, Decoding=argmax2019.0929.34
FlowSeq-baseKnowledge Distillation=false, Decoding=argmax2019.0929.26
Super (11B)Architecture=T5-11B, FLOPs=87e112022.0328.9
E-LANGArchitecture=T5, Super model=T5-11B, Swift model=T5-Large, Avg. Speed-up (FLOPs)=1.4X, Avg. Speed-up (time)=1.1X2022.0328.9
Swift (Large)Architecture=T5-Large, FLOPs=4.25e112022.0328.6
HiWi for Bias#Tuned=4.7%, #Stored=0.02%2023.0528
NAT w/ FTKnowledge Distillation=true, Decoding=argmax2019.0927.29
BitFit#Tuned=0.05%, #Stored=0.05%2023.0526.4
NAT-IRKnowledge Distillation=true, Decoding=argmax2019.0924.45
CMLM-baseKnowledge Distillation=true, Decoding=argmax2019.0923.65
Masked Label Smoothing (MLS)Vocabulary Sharing=true2022.0320.88
TransformerVocabulary Sharing=false, Label Smoothing=true2022.0320.68
TransformerVocabulary Sharing=true, Label Smoothing=true2022.0320.59
CMLM-baseKnowledge Distillation=false, Decoding=argmax2019.0920.24
CMLM-smallKnowledge Distillation=true, Decoding=argmax2019.0920.12
CTC LossKnowledge Distillation=true, Decoding=argmax2019.0919.93
TransformerVocabulary Sharing=true, Label Smoothing=false2022.0319.91
TransformerVocabulary Sharing=false, Label Smoothing=false2022.0319.61