Language Modeling on text8
1.02BPCGPT-2 1.5B
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-2 1.5Bzero-shot=true, parameters=1.5B2023.05 | 1.02 | |
| GPT-2 762Mzero-shot=true, parameters=762M2023.05 | 1.06 | |
| GPT-2 345Mzero-shot=true, parameters=345M2023.05 | 1.11 | |
| Plaid 1Bzero-shot=true, parameters=1.3B2023.05 | 1.12 | |
| GPT-2 124Mzero-shot=true, parameters=124M2023.05 | 1.22 | |
| Discrete FlowType=Autoregressive Backbone2023.10 | 1.23 | |
| AutoregressiveType=Autoregressive Backbone2023.10 | 1.23 | |
| GPT-2Architecture=Autoregressive2024.12 | 1.23 | |
| Dirichlet-Categorical PMM2024.12 | 1.29 | |
| GenMD42024.12 | 1.34 | |
| AR Argmax FlowType=Autoregressive Backbone2023.10 | 1.39 | |
| SEDD AbsorbType=Ours (NAR)2023.10 | 1.39 | |
| SEDDTransition Type=Absorb2024.12 | 1.39 | |
| MACType=Non-autoregressive2023.10 | 1.4 | |
| BFNType=Non-autoregressive2023.10 | 1.41 | |
| Bayesian Flow Nets2024.12 | 1.41 | |
| D3PM AbsorbType=Non-autoregressive2023.10 | 1.45 | |
| D3PMTransition Type=Absorb2024.12 | 1.45 | |
| SEDD UniformType=Ours (NAR)2023.10 | 1.47 | |
| D3PM UniformType=Non-autoregressive2023.10 | 1.61 | |
| D3PMTransition Type=Uniform2024.12 | 1.61 | |
| Mult. DiffusionType=Non-autoregressive2023.10 | 1.72 | |
| IAF/SCFType=Autoregressive Backbone2023.10 | 1.88 |