Language Modeling on Penn Treebank medium (val)
81.9PerplexityVariational LSTM (medium, MC)
Evaluation Results
| Method | Links | |
|---|---|---|
| Variational LSTM (medium, MC)Parameters=20M, Setup=medium, Monte Carlo dropout=true2016.11 | 81.9 | |
| QRNN + zoneout (p = 0.1) (medium)Parameters=18M, Setup=medium, Zoneout ratio=0.1, Embedding dropout=0.5, Inter-layer dropout=0.52016.11 | 82.1 | |
| QRNN (medium)Parameters=18M, Setup=medium, Embedding dropout=0.5, Inter-layer dropout=0.52016.11 | 82.9 | |
| Zoneout + Variational LSTM (medium)Parameters=20M, Setup=medium2016.11 | 84.4 | |
| LSTM (medium)Parameters=20M, Setup=medium, Implementation=Our models2016.11 | 85.7 | |
| LSTM (medium)Parameters=20M, Setup=medium2016.11 | 86.2 |