Language Modeling on LAMBADA (control)
94PerplexityLSTM-1024
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LSTM-1024Hidden units=1024, Implementation=Current paper reproduction2016.12 | 94 | — | |
| Neural cache model2016.12 | 129 | — | |
| Neural Cache2018.07 | 129 | — | |
| Universal Transformersteps=8, halting=fixed2018.07 | 129 | 32 | |
| Universal Transformersteps=9, halting=fixed2018.07 | 129 | 33 | |
| Universal Transformerhalting=dynamic2018.07 | 130 | 32 | |
| Universal Transformersteps=6, halting=fixed2018.07 | 131 | 32 | |
| LSTM2018.07 | 138 | 23 | |
| Transformer2018.07 | 142 | 19 | |
| LSTM-512Hidden units=5122016.12 | 149 | — | |
| WB5+cacheDescription=5-gram language model with cache2016.12 | 270 | — | |
| WB5Description=5-gram language model with Witten-Bell smoothing2016.12 | 285 | — |