Language Modeling on Penn Treebank medium (test)
78.3PerplexityQRNN + zoneout (p = 0.1) (medium)
Evaluation Results
| Method | Links | |
|---|---|---|
| QRNN + zoneout (p = 0.1) (medium)Parameters=18M, Setup=medium, Zoneout ratio=0.1, Embedding dropout=0.5, Inter-layer dropout=0.52016.11 | 78.3 | |
| LSTM with CharCNN embeddingsParameters=19M2016.11 | 78.9 | |
| Variational LSTM (medium, MC)Parameters=20M, Setup=medium, Monte Carlo dropout=true2016.11 | 79.7 | |
| QRNN (medium)Parameters=18M, Setup=medium, Embedding dropout=0.5, Inter-layer dropout=0.52016.11 | 79.9 | |
| Zoneout + Variational LSTM (medium)Parameters=20M, Setup=medium2016.11 | 80.6 | |
| LSTM (medium)Parameters=20M, Setup=medium, Implementation=Our models2016.11 | 82 | |
| LSTM (medium)Parameters=20M, Setup=medium2016.11 | 82.7 |