Language Modeling on BILLION WORD (test)
23.5PerplexitySRU++
Evaluation results
| Method | Links | |
|---|---|---|
| SRU++Parameters=465M, Effective GPU days=63, k=5, Mixed precision training=true2021.02 | 23.5 | |
| TransformerParameters=465M, Effective GPU days=192, Mixed precision training=true2021.02 | 23.9 | |
| SRU++Parameters=328M, Effective GPU days=36, Mixed precision training=true2021.02 | 25.1 | |
| TransformerEffective GPU days=147, Mixed precision training=true2021.02 | 25.2 | |
| TransformerParameters=331M, Effective GPU days=57, Mixed precision training=true2021.02 | 25.6 |