Neural Machine Translation on WMT14 English-to-German big (test)
30.1BLEUPre-Norm Transformer
Evaluation Results
| Method | Links | |
|---|---|---|
| Pre-Norm TransformerArchitecture=Big, Normalization=PN2020.03 | 30.1 | |
| Dynamic ConvArchitecture=Big2020.03 | 29.7 | |
| Transformer + LayerDropArchitecture=Big2020.03 | 29.6 | |
| Pre-Norm TransformerArchitecture=Big, Normalization=LN2020.03 | 29.5 | |
| Fixup-InitArchitecture=Big2020.03 | 29.3 | |
| Scaling NMTArchitecture=Big2020.03 | 29.3 | |
| DS-InitArchitecture=Big2020.03 | 29.1 | |
| Pre-Norm TransformerArchitecture=Big, Normalization=PN-V2020.03 | 28.5 | |
| TransformerArchitecture=Big2020.03 | 28.4 | |
| Pre-Norm TransformerArchitecture=Big, Normalization=BN2020.03 | 28.1 |