Language Modeling on PTB English Mikolov preprocessed (val)
44.9PerplexityMogrifier
Evaluation Results
| Method | Links | |
|---|---|---|
| MogrifierModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=true2019.09 | 44.9 | |
| MogrifierModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 45.1 | |
| FRAGEModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=22M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 47.4 | |
| AWD-LSTMModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=22M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 48.3 | |
| LSTMModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=true2019.09 | 48.6 | |
| LSTMModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 48.9 | |
| MogrifierModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=true2019.09 | 51.4 | |
| MogrifierModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 52.1 | |
| FRAGEModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=22M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 54.1 | |
| LSTMModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=true2019.09 | 55.5 | |
| LSTMModel Depth (dN)=2, Number of Parameters=24M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 55.8 | |
| AWD-LSTMModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=22M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 56.5 | |
| Transformer-XLNumber of Parameters=24M, Dynamic Evaluation=No2019.09 | 56.7 |