Machine Translation on WMT14 En-De newstest2014 (test)
30.4BLEUMask Attention Networks
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mask Attention NetworksModel size=big, Parameters=215M2021.03 | 30.4 | — | — | |
| DynamicConvParam (En-De)=213M2019.01 | 29.7 | — | — | |
| Dynamic ConvModel size=big, Parameters=213M2021.03 | 29.7 | — | — | |
| Ott et al. (2018)Param (En-De)=210M2019.01 | 29.3 | — | — | |
| Scaling NMTModel size=big, Parameters=213M2021.03 | 29.3 | — | — | |
| Shaw et al. (2018)2019.01 | 29.2 | — | — | |
| Local TransformerModel size=big, Parameters=268M2021.03 | 29.2 | — | — | |
| Relative TransformerModel size=big2021.03 | 29.2 | — | — | |
| Mask Attention NetworksModel size=base, Parameters=63M2021.03 | 29.1 | — | — | |
| Ahmed et al. (2017)Param (En-De)=213M2019.01 | 28.9 | — | — | |
| LightConvParam (En-De)=202M2019.01 | 28.9 | — | — | |
| Weighted TransformerModel size=big, Parameters=213M2021.03 | 28.9 | — | — | |
| Convolutional TransformerModel size=big2021.03 | 28.7 | — | — | |
| RevCol-TransformerEncoder arch=B=(1,1,1,1), COL=4, Encoder dmodel=768, Encoder dff=3072, Encoder head=12, Decoder arch=N=6, Decoder dmodel=768, Decoder dff=3072, Decoder head=12, Params=200M2022.12 | 28.67 | — | — | |
| Chen et al. (2018)Param (En-De)=379M2019.01 | 28.5 | — | — | |
| Local TransformerModel size=base, Parameters=89M2021.03 | 28.5 | — | — | |
| Transformer bigEncoder arch=N=6, Encoder dmodel=1024, Encoder dff=4096, Encoder head=16, Decoder arch=N=6, Decoder dmodel=1024, Decoder dff=4096, Decoder head=16, Params=209M2022.12 | 28.43 | — | — | |
| Vaswani et al. (2017)Param (En-De)=213M2019.01 | 28.4 | — | — | |
| TransformerModel size=big, Parameters=213M2021.03 | 28.4 | — | — | |
| Weighted TransformerModel size=base, Parameters=65M2021.03 | 28.4 | — | — | |
| Convolutional TransformerModel size=base, Parameters=88M2021.03 | 28.2 | — | — | |
| JM-NATIdec=102020.08 | 27.31 | — | 5.7 | |
| TransformerModel size=base, Parameters=62M2021.03 | 27.3 | — | — | |
| Transformertype=AT2020.08 | 27.3 | — | — | |
| LevTIdec=6+2020.08 | 27.27 | — | 4 | |
| Mask-PredictIdec=10, type=Iterative NAT2020.08 | 27.03 | — | 1.7 | |
| Relative TransformerModel size=base2021.03 | 26.8 | — | — | |
| GLAT + NPDIdec=1, m=72020.08 | 26.55 | — | 7.9 | |
| GLAT + CTCIdec=12020.08 | 26.39 | — | 14.6 | |
| LaNMTIdec=42020.08 | 26.3 | — | 5.7 | |
| NAT-DCRFIdec=1, NPD=true, m=92020.08 | 26.07 | — | 6.1 | |
| ImputerIdec=1, CTC=true2020.08 | 25.8 | — | 18.6 | |
| CTC*Idec=1, implementation=ours2020.08 | 25.52 | — | 14.6 | |
| FlowseqIdec=1, NPD=true, m=302020.08 | 25.31 | — | — | |
| GLATIdec=12020.08 | 25.21 | — | 15.3 | |
| Gehring et al. (2017)Param (En-De)=216M2019.01 | 25.2 | — | — | |
| NAT-HINTIdec=1, NPD=true, m=92020.08 | 25.2 | — | — | |
| imit-NATIdec=1, NPD=true, m=72020.08 | 24.15 | — | 9.7 | |
| FlowseqIdec=12020.08 | 23.72 | — | 1.1 | |
| Autoregressivebeam size=42017.11 | 23.45 | 607 | 1 | |
| NAT-DCRFIdec=12020.08 | 23.44 | — | 10.4 | |
| Base BPE-level NMT (Row c)Target=BPE, Depth=2, Attention=Concatenated, Model=Base, Ensemble=true2016.03 | 23.11 | — | — | |
| Base BPE-level NMT (Row b)Target=BPE, Depth=2, Attention=hL, Model=Base, Ensemble=true2016.03 | 23.1 | — | — | |
| Base Character-level NMT (Row e)Target=Char, Depth=2, Attention=Concatenated, Model=Base, Ensemble=true2016.03 | 23.04 | — | — | |
| Autoregressivebeam size=12017.11 | 22.71 | 408 | 1.49 | |
| imit-NATIdec=12020.08 | 22.44 | — | 18.6 | |
| NAT-IRIdec=102020.08 | 21.61 | — | 1.5 | |
| Base BPE-level NMT (Row c)Target=BPE, Depth=2, Attention=Concatenated, Model=Base, Ensemble=false2016.03 | 21.33 | — | — | |
| Base Character-level NMT (Row e)Target=Char, Depth=2, Attention=Concatenated, Model=Base, Ensemble=false2016.03 | 21.25 | — | — | |
| NAT-HINTIdec=12020.08 | 21.11 | — | — | |
| State-of-the-art Non-Neural ApproachApproach=Non-Neural2016.03 | 20.6 | — | — | |
| Base BPE-level NMT (Row b)Target=BPE, Depth=2, Attention=hL, Model=Base, Ensemble=false2016.03 | 20.47 | — | — | |
| NAT-base*Idec=1, implementation=ours2020.08 | 20.36 | — | 15.3 | |
| Bi-scale Character-level NMT (Row f)Target=Char, Depth=2, Attention=hL, Model=Bi-S, Ensemble=false2016.03 | 20.19 | — | — | |
| Base BPE-level NMT (Row a)Target=BPE, Depth=1, Attention=h1, Model=Base, Ensemble=false2016.03 | 19.98 | — | — | |
| Base Character-level NMT (Row d)Target=Char, Depth=2, Attention=hL, Model=Base, Ensemble=false2016.03 | 19.7 | — | — | |
| Bi-scale Character-level NMT (Row g)Target=Char, Depth=2, Attention=Concatenated, Model=Bi-S, Ensemble=false2016.03 | 19.39 | — | — | |
| NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=1002017.11 | 19.17 | 257 | 2.36 | |
| NAT-FTIdec=1, NPD=true, m=1002020.08 | 19.17 | — | 2.4 | |
| NATdecoding=Noisy Parallel Decoding, fine-tuning=true, sample size=102017.11 | 18.66 | 79 | 7.68 | |
| Mask-PredictIdec=1, type=Fully NAT2020.08 | 18.05 | — | — | |
| NATdecoding=argmax, fine-tuning=true2017.11 | 17.69 | 39 | 15.6 | |
| NAT-FTIdec=12020.08 | 17.69 | — | 15.6 | |
| NATdecoding=argmax2017.11 | 17.35 | 39 | 15.6 | |
| NAT-CTCIdec=1, CTC=true2020.08 | 16.56 | — | — |