Autoregressive Language Modeling on LM1B
21.5PPLOmniNet_T (Large)
Evaluation results
| Method | Links | |
|---|---|---|
| OmniNet_T (Large)#Params=0.1B2021.03 | 21.5 | |
| OmniNet_P (Large)#Params=0.1B2021.03 | 21.6 | |
| Transformer-XL#Params=0.8B2021.03 | 21.8 | |
| OmniNet_B (Large)#Params=0.1B2021.03 | 22 | |
| Transformer-XL#Params=0.5B2021.03 | 23.5 | |
| Adaptive Input#Params=1.0B2021.03 | 23.7 | |
| Adaptive Input#Params=0.5B2021.03 | 24.1 |