Machine Translation on WMT en-fr 14
45BLEU ScoreOracle Rescoring
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Oracle RescoringInference Protocol=Rescoring of the Baseline 1000-best lists2014.09 | 45 | — | |
| MUSEModel Size=Large2019.11 | 43.5 | — | |
| Local Joint Self-attention2019.11 | 43.3 | — | |
| Transformer (Ott et al., 2018)2019.11 | 43.2 | — | |
| DynamicConv2019.11 | 43.2 | — | |
| MUSE-simpleModel Size=Large2019.11 | 43.2 | — | |
| OmniNetvariant=P2021.03 | 42.6 | — | |
| 60L TransformerLayers=602021.03 | 41.8 | — | |
| Transformer (relative position)Attention=Relative Position2019.11 | 41.5 | — | |
| Weighted Transformer2019.11 | 41.4 | — | |
| Large TransformerSize=Large2021.03 | 41.4 | — | |
| Evolved Transformer2019.11 | 41.3 | — | |
| GNMT (RL-refined WPM-32K)ensemble=8 models, refinement=Reinforcement Learning (RL), vocabulary_size=32K2016.09 | 41.16 | — | |
| Transformer (Vaswani et al., 2017)2019.11 | 41 | — | |
| SM3Batch Size Per Core (Total)=24 (768), Memory Usage Per Core=7.02 GiB2019.01 | 40.5 | — | |
| ConvSeq2seq2019.11 | 40.5 | — | |
| Deep-Att + PosUnkensemble=8 models, architecture=Deep-Att, position_unknown_mapping=PosUnk2016.09 | 40.4 | — | |
| GNMT (WPM-32K)ensemble=8 models, vocabulary_size=32K2016.09 | 40.35 | — | |
| AdagradBatch Size Per Core (Total)=12 (384), Memory Usage Per Core=6.85 GiB2019.01 | 39.9 | — | |
| SM3Batch Size Per Core (Total)=12 (384), Memory Usage Per Core=5.36 GiB2019.01 | 39.81 | — | |
| AdafactorBatch Size Per Core (Total)=24 (768), Memory Usage Per Core=7.04 GiB2019.01 | 39.65 | — | |
| AdamBatch Size Per Core (Total)=12 (384), Memory Usage Per Core=6.88 GiB2019.01 | 38.96 | — | |
| AdafactorBatch Size Per Core (Total)=12 (384), Memory Usage Per Core=5.43 GiB2019.01 | 37.89 | — | |
| LSTM + PosUnklayers=6, position_unknown_mapping=PosUnk2016.09 | 37.5 | — | |
| Best WMT'14 result2014.09 | 37 | — | |
| LSTMInference Protocol=Rescoring the baseline 1000-best, Model Direction=reversed, Ensemble Size=52014.09 | 36.5 | — | |
| LSTMInference Protocol=Rescoring the baseline 1000-best, Model Direction=reversed, Ensemble Size=12014.09 | 35.85 | — | |
| LSTMInference Protocol=Rescoring the baseline 1000-best, Model Direction=forward, Ensemble Size=12014.09 | 35.61 | — | |
| LSTMlayers=62016.09 | 35.6 | — | |
| LSTM EnsembleArchitecture=reversed, ensemble size=5, beam size=122014.09 | 34.81 | — | |
| CSLM + RNNbaseline_configuration=true, CSLM=true, RNN Encoder-Decoder=true2014.06 | 34.64 | — | |
| CSLM + RNN + WPbaseline_configuration=true, CSLM=true, RNN Encoder-Decoder=true, Word Penalty=true2014.06 | 34.54 | — | |
| Cho et al.2014.09 | 34.54 | — | |
| LSTM EnsembleArchitecture=reversed, ensemble size=5, beam size=22014.09 | 34.5 | — | |
| RNNbaseline_configuration=true, RNN Encoder-Decoder=true2014.06 | 33.87 | — | |
| Baselinebaseline_configuration=true2014.06 | 33.3 | — | |
| Baseline Systemtype=SMT2014.09 | 33.3 | — | |
| Baseline System2014.09 | 33.3 | — | |
| LIUM systemtype=phrase-based SMT2014.09 | 33.3 | — | |
| Moses2014.09 | 33.3 | — | |
| LSTM EnsembleArchitecture=reversed, ensemble size=2, beam size=122014.09 | 33.27 | — | |
| LSTM EnsembleArchitecture=reversed, ensemble size=5, beam size=12014.09 | 33 | — | |
| CSLM + RNN + WPbaseline_configuration=true, CSLM=true, RNN Encoder-Decoder=true, Word Penalty=true2014.06 | 31.5 | — | |
| CSLM + RNNbaseline_configuration=true, CSLM=true, RNN Encoder-Decoder=true2014.06 | 31.48 | — | |
| RNNbaseline_configuration=true, RNN Encoder-Decoder=true2014.06 | 31.2 | — | |
| Baselinebaseline_configuration=true2014.06 | 30.64 | — | |
| LSTMArchitecture=reversed, ensemble size=1, beam size=122014.09 | 30.59 | — | |
| Regularized LSTMdropout=true, dropout_probability=0.2, hidden_layers=4, units=1000, beam_size=122014.09 | 29.03 | 5 | |
| Bahdanau et al.2014.09 | 28.45 | — | |
| RNNsearch-50*max_sentence_length=50, longer_training=true2014.09 | 28.45 | — | |
| RNNsearch-50max_sentence_length=502014.09 | 26.75 | — | |
| LSTMArchitecture=forward, ensemble size=1, beam size=122014.09 | 26.17 | — | |
| Non-regularized LSTMdropout=false, hidden_layers=4, units=1000, beam_size=122014.09 | 25.9 | 5.8 | |
| RNNsearch-30max_sentence_length=302014.09 | 21.5 | — | |
| RNNencdec-50max_sentence_length=502014.09 | 17.82 | — | |
| RNNencdec-30max_sentence_length=302014.09 | 13.93 | — |