Language Modeling on Wikitext-2 Standard (val)
40.2PerplexityMogrifier
Evaluation Results
| Method | Links | |
|---|---|---|
| MogrifierModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=true2019.09 | 40.2 | |
| MogrifierModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 40.6 | |
| FRAGEModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=35M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 40.8 | |
| AWD-LSTMModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=35M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 42.4 | |
| LSTMModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=false2019.09 | 43.2 | |
| LSTMModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=Yes, Monte-Carlo Dropout Evaluation=true2019.09 | 43.2 | |
| MogrifierModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=true2019.09 | 57.3 | |
| MogrifierModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 58.7 | |
| FRAGEModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=35M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 60.3 | |
| LSTMModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=true2019.09 | 61.9 | |
| LSTMModel Depth (dN)=2, Mixture of Softmaxes (MoS)=2, Number of Parameters=35M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 62.6 | |
| AWD-LSTMModel Depth (dN)=3, Mixture of Softmaxes (MoS)=15, Number of Parameters=35M, Dynamic Evaluation=No, Monte-Carlo Dropout Evaluation=false2019.09 | 63.9 |