Language Modeling on Penn Treebank (dev)
56.5Perplexity (PPL)AWD-LSTM-MoS + finetuning
Evaluation Results
| Method | Links | |
|---|---|---|
| AWD-LSTM-MoS + finetuning#Params=22M, finetuning=true2019.06 | 56.5 | |
| Transformer-XL#Params=24M2019.06 | 56.7 | |
| AWD-LSTM + Phrase Induction + finetuning#Params=24M, Phrase Induction=true, finetuning=true2019.06 | 57.8 | |
| ON-LSTM#Params=25M2019.06 | 58.3 | |
| Differentiable NAS#Params=23M2019.06 | 58.3 | |
| AWD-LSTM + Phrase Induction#Params=24M, Phrase Induction=true2019.06 | 59.6 | |
| AWD-LSTM + finetuning#Params=24M, finetuning=true2019.06 | 60 | |
| AWD-LSTM + Phrase Induction (No Attention)#Params=24M, Phrase Induction=true, attention=false2019.06 | 60.2 | |
| AWD-LSTM#Params=24M2019.06 | 60.7 | |
| Efficient NAS#Params=24M2019.06 | 60.8 | |
| 4-layer skip LSTM (tied)#Params=24M2019.06 | 60.9 | |
| AWD-LSTM + Phrase Induction (No NS)#Params=24M, Phrase Induction=true, negative sampling=false2019.06 | 61 | |
| Recurrent Highway Networks#Params=23M2019.06 | 67.9 | |
| Tied Variational LSTM#Params=24M2019.06 | 75.7 |