Language Modeling on Penn Treebank (PTB) word-level (val)
56.5PerplexityAWD-LSTM-MoS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AWD-LSTM-MoSParameters=22M2018.10 | 56.5 | — | |
| ON-LSTM - 3-layer (tied)Parameters=25M, tied=true2018.10 | 58.29 | — | |
| AWD-LSTM - 3-layer LSTM (tied)Parameters=24M, tied=true2018.10 | 60 | — | |
| 4-layer skip connection LSTM (tied)Parameters=24M, tied=true2018.10 | 60.9 | — | |
| Variational RHN (tied)Parameters=23M, tied=true2018.10 | 67.9 | — | |
| Variational LSTM (tied) + augmented lossParameters=51M, tied=true2018.10 | 71.1 | — | |
| Pointer Sentinel-LSTMParameters=21M2018.10 | 72.4 | — | |
| LSTM (large)Parameters=66M2018.10 | 82.2 | — | |
| LSTMParams=3.6M2025.11 | 165.55 | 65 | |
| INNParams=5.5M2025.11 | 207.2 | 107 | |
| TransformerParams=5.7M2025.11 | 210.45 | 111 |