Machine Translation on IWSLT DE-EN 2014 (train)
3Training PerplexityKnee schedule
Evaluation results
| Method | Links | |
|---|---|---|
| Knee scheduleLR Schedule=Knee schedule, Architecture=Transformer, Training Epochs=502020.03 | 3 | |
| Cosine DecayLR Schedule=Cosine Decay, Architecture=Transformer, Training Epochs=502020.03 | 3.08 | |
| CutoffLR Schedule=Knee, Training Epochs=1002020.03 | 3.29 | |
| Linear DecayLR Schedule=Linear Decay, Architecture=Transformer, Training Epochs=502020.03 | 3.36 | |
| CutoffLR Schedule=Inv. Sqrt, Training Epochs=1002020.03 | 3.46 | |
| CutoffLR Schedule=Knee (short budget), Training Epochs=702020.03 | 3.48 | |
| Knee scheduleLR Schedule=Knee schedule, Training Budget=35 epochs2020.03 | 3.58 | |
| One-CycleLR Schedule=One-Cycle, Architecture=Transformer, Training Epochs=502020.03 | 3.68 | |
| CutoffLR Schedule=Inv. Sqrt (short budget), Training Epochs=702020.03 | 3.76 | |
| Cosine DecayLR Schedule=Cosine Decay, Training Budget=35 epochs2020.03 | 3.86 | |
| One-CycleLR Schedule=One-Cycle, Training Budget=35 epochs2020.03 | 3.98 | |
| Linear DecayLR Schedule=Linear Decay, Training Budget=35 epochs2020.03 | 4.11 |