Character-level Language Modeling on Penn Treebank character-level (val)
1.24BPCLayerNorm HM-LSTM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LayerNorm HM-LSTM# layers=3, hidden / layer=512, Max dilations=12017.10 | 1.24 | — | — | — | |
| HyperNetworks# layers=1 / 2, hidden / layer=1k, # parameters (≈, M)=4.91 / 14.41, Max dilations=12017.10 | 1.26 | — | — | — | |
| Zoneout# layers=1, hidden / layer=1k, # parameters (≈, M)=4.25, Max dilations=12017.10 | 1.27 | — | — | — | |
| Dilated GRU# layers=5, hidden / layer=256, # parameters (≈, M)=1.42, Max dilations=642017.10 | 1.27 | — | — | — | |
| Recurrent dropout LSTM# layers=1, hidden / layer=1k, # parameters (≈, M)=4.25, Max dilations=12017.10 | 1.3 | — | — | — | |
| LSTM# layers=1 / 5, hidden / layer=1k / 256, # parameters (≈, M)=4.25 / 1.9, Max dilations=12017.10 | 1.31 | — | — | — | |
| Dilated LSTM# layers=5, hidden / layer=256, # parameters (≈, M)=1.9, Max dilations=642017.10 | 1.31 | — | — | — | |
| GRU# layers=1 / 5, hidden / layer=1k / 256, # parameters (≈, M)=3.19 / 1.42, Max dilations=12017.10 | 1.32 | — | — | — | |
| Recurrent BN-LSTM# layers=1, hidden / layer=1k, Max dilations=12017.10 | 1.32 | — | — | — | |
| Dilated Vanilla# layers=5, hidden / layer=256, # parameters (≈, M)=0.6, Max dilations=642017.10 | 1.37 | — | — | — | |
| BASE2026.05 | — | 1.0897 | — | — | |
| EGA-12026.05 | — | 0.9889 | 0.1009 | — |