Language Modeling on Google Billion Word (test)
26.67PerplexityDynamicConv
Evaluation results
| Method | Links | |
|---|---|---|
| DynamicConvParameters=339M2019.01 | 26.67 | |
| Self-attention baselineParameters=331M2019.01 | 26.73 | |
| Mixture of ExpertsParameters=4371M†2019.01 | 28 | |
| 2-layer LSTM-8192-10242019.01 | 30.6 | |
| Gated Convolutional ModelParameters=428M2019.01 | 31.9 |