Machine Translation on WMT English-to-German 2013 (test)
24.68BLEU ScoreLookahead (Adam)
Evaluation results
| Method | Links | |
|---|---|---|
| Lookahead (Adam)Inner optimizer=ADAM, Training steps=50k, Model architecture=Transformer Base, Learning rate warm-up=True2019.07 | 24.68 | |
| ADAMTraining steps=50k, Model architecture=Transformer Base, Learning rate warm-up=True2019.07 | 24.6 | |
| Lookahead (Adam without warm-up)Inner optimizer=ADAM, Training steps=50k, Model architecture=Transformer Base, Learning rate warm-up=False2019.07 | 24.3 | |
| AdafactorTraining steps=50k, Model architecture=Transformer Base, Learning rate warm-up=True2019.07 | 24.17 |