Machine Translation on WMT'14 EN-DE (val)
4.87Validation PerplexityKnee schedule
Evaluation results
| Method | Links | |
|---|---|---|
| Knee scheduleArchitecture=Transformer networks, Training Epochs=70, LR Schedule=Knee schedule2020.03 | 4.87 | |
| Linear DecayArchitecture=Transformer networks, Training Epochs=70, LR Schedule=Linear Decay2020.03 | 4.89 | |
| Cosine DecayArchitecture=Transformer networks, Training Epochs=70, LR Schedule=Cosine Decay2020.03 | 4.91 | |
| Knee scheduleLR Schedule=Knee schedule, Training Epochs=30, Architecture=Transformer networks2020.03 | 4.92 | |
| One-CycleArchitecture=Transformer networks, Training Epochs=70, LR Schedule=One-Cycle2020.03 | 4.95 | |
| Cosine DecayLR Schedule=Cosine Decay, Training Epochs=30, Architecture=Transformer networks2020.03 | 4.99 | |
| One-CycleLR Schedule=One-Cycle, Training Epochs=30, Architecture=Transformer networks2020.03 | 5.02 | |
| Linear DecayLR Schedule=Linear Decay, Training Epochs=30, Architecture=Transformer networks2020.03 | 5.02 |