Word-level Prediction on PTB (test)
72.8PerplexityLEM
Evaluation results
| Method | Links | |
|---|---|---|
| LEM# units=256, # params=524k2021.10 | 72.8 | |
| LSTM# of layers=3, hidden size=7002019.07 | 78.93 | |
| R-Transformer# of layers=3, hidden size=1282019.07 | 84.38 | |
| TCN# of layers=4, hidden size=6002019.07 | 88.68 | |
| GRU2019.07 | 92.48 | |
| TARNN# units=256, # params=524k2021.10 | 94.6 | |
| SkipLSTM# units=256, # params=524k2021.10 | 114.2 | |
| RNN2019.07 | 114.5 | |
| Lipschitz RNN using EulerN=160, # params=50K2020.06 | 115.36 | |
| Lipschitz RNN# units=160, # params=76k2021.10 | 115.4 | |
| Incremental RNN# params=29.5K2020.06 | 115.71 | |
| FastRNN# units=256, # params=131k2021.10 | 115.9 | |
| FastGRNN-LSQ# params=52.5K2020.06 | 115.92 | |
| FastGRNN# params=52.5K2020.06 | 116.11 | |
| LSTM# units=256, # params=524k2021.10 | 116.9 | |
| LSTM# params=210K2020.06 | 117.41 | |
| Transformer# of layers=3, hidden size=1282019.07 | 122.37 | |
| FastRNN# params=52.5K2020.06 | 127.76 | |
| SpectralRNN# params=24.8K2020.06 | 130.2 |