Machine Translation on WMT EN-DE '14 (train)
3.87Training PerplexityCosine Decay
Evaluation results
| Method | Links | |
|---|---|---|
| Cosine DecayArchitecture=Transformer networks, Training Epochs=70, LR Schedule=Cosine Decay2020.03 | 3.87 | |
| Linear DecayArchitecture=Transformer networks, Training Epochs=70, LR Schedule=Linear Decay2020.03 | 3.87 | |
| Knee scheduleArchitecture=Transformer networks, Training Epochs=70, LR Schedule=Knee schedule2020.03 | 3.89 | |
| One-CycleArchitecture=Transformer networks, Training Epochs=70, LR Schedule=One-Cycle2020.03 | 3.96 | |
| Knee scheduleLR Schedule=Knee schedule, Training Epochs=30, Architecture=Transformer networks2020.03 | 4.31 | |
| Cosine DecayLR Schedule=Cosine Decay, Training Epochs=30, Architecture=Transformer networks2020.03 | 4.32 | |
| Linear DecayLR Schedule=Linear Decay, Training Epochs=30, Architecture=Transformer networks2020.03 | 4.36 | |
| One-CycleLR Schedule=One-Cycle, Training Epochs=30, Architecture=Transformer networks2020.03 | 4.38 |