Machine Translation on IWSLT German-to-English '14 (test)
37.6BLEU ScoreTransformer Base & Cutoff
Evaluation Results
| Method | Links | |
|---|---|---|
| Transformer Base & CutoffBackbone=Transformer Base, JS loss=true2020.09 | 37.6 | |
| Data Diversification2020.09 | 37.2 | |
| Transformer Base & CutoffBackbone=Transformer Base, JS loss=false2020.09 | 36.7 | |
| MAT+Knee2020.09 | 36.6 | |
| Mixed Representations2020.09 | 36.4 | |
| MAT2020.09 | 36.2 | |
| AGDArchitecture=Transformer2023.12 | 35.94 | |
| AdaBeliefArchitecture=Transformer2023.12 | 35.93 | |
| AdamWArchitecture=Transformer2023.12 | 35.82 | |
| AdaHessianArchitecture=Transformer2023.12 | 35.79 | |
| SGD temperatureArchitecture=6L-Encoder / 6L-Decoder, Optimizer=SGD, Scaling=inverse temperature2022.06 | 35.69 | |
| T-FixupArchitecture=6L-Encoder / 6L-Decoder2022.06 | 35.59 | |
| Adam POST-LNArchitecture=6L-Encoder / 6L-Decoder, Optimizer=Adam2022.06 | 35.39 | |
| Adversarial training2020.09 | 35.2 | |
| Adam PRE-LNArchitecture=6L-Encoder / 6L-Decoder, Optimizer=Adam2022.06 | 35.1 | |
| APO-PrecondModel=Transformer2022.02 | 34.62 | |
| AdamWModel=Transformer2022.02 | 34.6 | |
| ReZeroArchitecture=6L-Encoder / 6L-Decoder2022.06 | 34.55 | |
| Transformer Base2020.09 | 34.4 | |
| Transformerembedding_method=FRAGE2018.09 | 33.97 | |
| AdamWimplementation=Fairseq2021.03 | 33.95 | |
| SGDHessimplementation=Fairseq2021.03 | 33.73 | |
| Variational Attentioninference_type=Enum2018.07 | 33.69 | |
| AdaHessianimplementation=Fairseq2021.03 | 33.62 | |
| Variational Attentioninference_type=Sample2018.07 | 33.3 | |
| Marginal Likelihood2018.07 | 33.29 | |
| Transformer2018.09 | 33.12 | |
| AdamArchitecture=Transformer2023.12 | 32.93 | |
| ConvS2S+Risk2018.09 | 32.93 | |
| Minimum Risk Training2018.07 | 32.84 | |
| SGD res-scaleArchitecture=6L-Encoder / 6L-Decoder, Optimizer=SGD, Residual Scaling=depth-dependent2022.06 | 32.79 | |
| Soft Attention2018.07 | 32.77 | |
| ConvS2S+SeqNLL2018.09 | 32.68 | |
| Dual transfer learning2018.09 | 32.35 | |
| DenseNMT-8L-2 (BPE)Decoding Strategy=Beam, Preprocessing=BPE2018.06 | 32.26 | |
| DenseNMT-8L-1 (BPE)Decoding Strategy=Beam, Preprocessing=BPE2018.06 | 32.08 | |
| SGDmModel=Transformer2022.02 | 31.43 | |
| Hard Attentioninference_type=Enum2018.07 | 31.4 | |
| SGD POST-LNArchitecture=6L-Encoder / 6L-Decoder, Optimizer=SGD2022.06 | 31.36 | |
| DenseNMT-8L-2 (BPE)Decoding Strategy=Greedy, Preprocessing=BPE2018.06 | 30.8 | |
| DenseNMT-8L-1 (BPE)Decoding Strategy=Greedy, Preprocessing=BPE2018.06 | 30.5 | |
| Hard Attentioninference_type=Sample2018.07 | 30.42 | |
| DenseNMT-8L-2 (word)Decoding Strategy=Beam, Preprocessing=word2018.06 | 30.33 | |
| NPMT+LM (this paper)Decoding strategy=Beam Search2017.06 | 30.08 | |
| Neural PBMT + LM2018.07 | 30.08 | |
| Variational Relaxed Attention2018.07 | 30.05 | |
| DeepConv2018.09 | 30.04 | |
| NPMT (this paper)Decoding strategy=Beam Search2017.06 | 29.92 | |
| SGDimplementation=Fairseq2021.03 | 29.75 | |
| NPMT+LMDecoding Strategy=Beam2018.06 | 29.16 | |
| DenseNMT-8L-2 (word)Decoding Strategy=Greedy, Preprocessing=word2018.06 | 29.11 | |
| NPMTDecoding Strategy=Beam2018.06 | 28.96 | |
| SGDArchitecture=Transformer2023.12 | 28.57 | |
| NPMT (this paper)Decoding strategy=Greedy2017.06 | 28.57 | |
| ACDecoding Strategy=Beam2018.06 | 28.53 | |
| AC+LL (Bahdanau et al., 2017)Decoding strategy=Beam Search2017.06 | 28.53 | |
| Actor-Critic2018.07 | 28.53 | |
| Actor-critic2020.09 | 28.5 | |
| RF-C+LL (Bahdanau et al., 2017)Decoding strategy=Beam Search2017.06 | 28.3 | |
| NPMTDecoding Strategy=Greedy2018.06 | 27.83 | |
| RF-C+LL (Bahdanau et al., 2017)Decoding strategy=Greedy2017.06 | 27.7 | |
| LL*Decoding strategy=Beam Search2017.06 | 27.61 | |
| LL (Bahdanau et al., 2017)Decoding strategy=Beam Search2017.06 | 27.56 | |
| ACDecoding Strategy=Greedy2018.06 | 27.49 | |
| AC+LL (Bahdanau et al., 2017)Decoding strategy=Greedy2017.06 | 27.49 | |
| BSO, SB-ΔKte=5, Ktr=62016.06 | 26.36 | |
| Beam Search Optimization2018.07 | 26.36 | |
| LL*Decoding strategy=Greedy2017.06 | 26.17 | |
| LL (Bahdanau et al., 2017)Decoding strategy=Greedy2017.06 | 25.82 | |
| BSO, SB-ΔKte=10, Ktr=62016.06 | 25.48 | |
| BSO (Wiseman & Rush, 2016)Decoding strategy=Beam Search2017.06 | 25.48 | |
| NPMT-LMSearch Strategy=Beam Search2017.06 | 25.36 | |
| NPMTSearch Strategy=Beam Search2017.06 | 25.08 | |
| seq2seqKte=52016.06 | 24.03 | |
| Word-level CNN w/attn, input feedingGranularity=Word-level, Attention=With, Input feeding=With2016.11 | 24 | |
| seq2seqKte=102016.06 | 23.87 | |
| LL (Wiseman & Rush, 2016)Decoding strategy=Beam Search2017.06 | 23.87 | |
| BSO, SB-ΔKte=1, Ktr=62016.06 | 23.83 | |
| BSO (Wiseman & Rush, 2016)Decoding strategy=Greedy2017.06 | 23.83 | |
| NPMTSearch Strategy=Greedy2017.06 | 23.62 | |
| Sequence-to-sequence with attentionSearch Strategy=Beam Search2017.06 | 22.59 | |
| REINFORCE-criticDecoding method=beam search2016.07 | 22.58 | |
| seq2seqKte=12016.06 | 22.53 | |
| LL (Wiseman & Rush, 2016)Decoding strategy=Greedy2017.06 | 22.53 | |
| Actor-CriticDecoding method=beam search2016.07 | 22.45 | |
| DADKte=102016.06 | 22.4 | |
| DADKte=52016.06 | 22.25 | |
| REINFORCE-criticDecoding method=greedy search2016.07 | 22.24 | |
| MIXERDecoding Strategy=Beam2018.06 | 21.83 | |
| MIXERKte=102016.06 | 21.83 | |
| MIXER (Ranzato et al., 2015)Decoding strategy=Beam Search2017.06 | 21.83 | |
| MIXERKte=52016.06 | 21.81 | |
| Actor-CriticDecoding method=greedy search2016.07 | 21.66 | |
| LLDecoding method=beam search2016.07 | 21.46 | |
| REINFORCEDecoding method=beam search2016.07 | 21.35 | |
| Sequence-to-sequence with attentionSearch Strategy=Greedy2017.06 | 21.26 | |
| REINFORCEDecoding method=greedy search2016.07 | 20.92 | |
| MIXERDecoding Strategy=Greedy2018.06 | 20.73 | |
| MIXERKte=12016.06 | 20.73 | |
| MIXERDecoding method=greedy search, Reference=Ranzato et al., 20152016.07 | 20.73 |