Language Modeling on Text8 word-level (test)
99.9PerplexityNeural cache model
Evaluation Results
| Method | Links | |
|---|---|---|
| Neural cache modelbackbone=LSTM-10242016.12 | 99.9 | |
| LSTM-1024hidden units=1024, backbone=LSTM, implementation=our implementation2016.12 | 121.8 | |
| MemN2N# of hidden=500, # of hops=7, memory size=1002015.03 | 147 | |
| MemNNbackbone=MemNN2016.12 | 147 | |
| LSTM# of hidden=5002015.03 | 154 | |
| MemN2N# of hidden=500, # of hops=5, memory size=1002015.03 | 154 | |
| MemN2N# of hidden=500, # of hops=6, memory size=1502015.03 | 154 | |
| MemN2N# of hidden=500, # of hops=6, memory size=1002015.03 | 155 | |
| LSTM-500hidden units=500, backbone=LSTM2016.12 | 156 | |
| MemN2N# of hidden=500, # of hops=6, memory size=1252015.03 | 157 | |
| MemN2N# of hidden=500, # of hops=6, memory size=752015.03 | 158 | |
| SCRN# of hidden=5002015.03 | 161 | |
| SCRNNbackbone=SCRNN2016.12 | 161 | |
| MemN2N# of hidden=500, # of hops=4, memory size=1002015.03 | 162 | |
| MemN2N# of hidden=500, # of hops=6, memory size=252015.03 | 163 | |
| MemN2N# of hidden=500, # of hops=6, memory size=502015.03 | 166 | |
| MemN2N# of hidden=500, # of hops=3, memory size=1002015.03 | 178 | |
| RNN# of hidden=5002015.03 | 184 | |
| MemN2N# of hidden=500, # of hops=2, memory size=1002015.03 | 187 |