List your company
ResearchFormerly WizWand

Language Modeling on text8

1.02BPCGPT-2 1.5B
0.98561.21781.451.6822May 30, 2023Sep 1, 2023Dec 5, 2023Mar 8, 2024Jun 11, 2024Sep 13, 2024Dec 17, 2024
Updated 8mo ago

Evaluation results

MethodLinks
GPT-2 1.5Bzero-shot=true, parameters=1.5B2023.051.02
GPT-2 762Mzero-shot=true, parameters=762M2023.051.06
GPT-2 345Mzero-shot=true, parameters=345M2023.051.11
Plaid 1Bzero-shot=true, parameters=1.3B2023.051.12
GPT-2 124Mzero-shot=true, parameters=124M2023.051.22
Discrete FlowType=Autoregressive Backbone2023.101.23
AutoregressiveType=Autoregressive Backbone2023.101.23
GPT-2Architecture=Autoregressive2024.121.23
Dirichlet-Categorical PMM2024.121.29
GenMD42024.121.34
AR Argmax FlowType=Autoregressive Backbone2023.101.39
SEDD AbsorbType=Ours (NAR)2023.101.39
SEDDTransition Type=Absorb2024.121.39
MACType=Non-autoregressive2023.101.4
BFNType=Non-autoregressive2023.101.41
Bayesian Flow Nets2024.121.41
D3PM AbsorbType=Non-autoregressive2023.101.45
D3PMTransition Type=Absorb2024.121.45
SEDD UniformType=Ours (NAR)2023.101.47
D3PM UniformType=Non-autoregressive2023.101.61
D3PMTransition Type=Uniform2024.121.61
Mult. DiffusionType=Non-autoregressive2023.101.72
IAF/SCFType=Autoregressive Backbone2023.101.88