Machine Translation on Google Production En→Fr (Eval)
2.6PerplexityMoE with 2048 Experts
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MoE with 2048 Expertsops/timestep=85M, Total #Parameters=8.7B, Training Time=1 day/64 k40s2017.01 | 2.6 | 37.27 | |
| GNMTops/timestep=214M, Total #Parameters=278M, Training Time=6 days/96 k80s2017.01 | 2.78 | 35.8 |