Language Modeling on WikiText-2 v1 (val)
42.41PerplexityAWD-LSTM-MoS + dynamical evaluation
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AWD-LSTM-MoS + dynamical evaluation#Param=35M, dynamic evaluation=true2019.05 | 42.41 | — | |
| AWD-LSTM-DRILL + dynamic evaluation#Param=34M, dynamic evaluation=true2019.05 | 43.9 | — | |
| AWD-LSTM + dynamic evaluation#Param=33M, dynamic evaluation=true2019.05 | 46.4 | — | |
| AWD-LSTM + continuous cache pointer#Param=33M, dynamic evaluation=true2019.05 | 53.8 | — | |
| Ensemble of AllWeight=12023.11 | 55.4 | — | |
| AWD-LSTM-DOCWeight=0.412023.11 | 60.27 | — | |
| AWD-LSTM-MoS#Param=35M2019.05 | 63.88 | — | |
| AWD-LSTM-MOSWeight=0.172023.11 | 63.92 | — | |
| AWD-LSTM-DRILL#Param=34M, finetuned=true2019.05 | 64.9 | — | |
| AWD-LSTM-DRILL#Param=34M, finetuned=false2019.05 | 65.7 | — | |
| Transformer-XLWeight=0.162023.11 | 67.41 | — | |
| AWD-LSTM#Param=33M, finetuned=true2019.05 | 68.6 | — | |
| 2-layer skip connection LSTM#Param=24M2019.05 | 69.1 | — | |
| AWD-LSTM#Param=33M, finetuned=false2019.05 | 69.1 | — | |
| QRNNWeight=0.002023.11 | 69.23 | — | |
| EGRUWeight=0.222023.11 | 69.4 | — | |
| Fast WeightsWeight=0.032023.11 | 69.51 | — | |
| Variational LSTM + augmented loss#Param=28M2019.05 | 91.5 | — | |
| Fuzzy-Gated + RPAcontext=768, evaluation protocol=sequential, stride=no-overlap (stride=context)2026.03 | 189.8 | 5.246 | |
| KnerserNey-5gramWeight=0.012023.11 | 233.93 | — |