List your company
ResearchFormerly WizWand

Language Modeling on LM1B zero-shot

51.25PerplexityAR
48.611666.420884.23102.0392Jun 11, 2024Oct 10, 2024Feb 8, 2025Jun 9, 2025Oct 8, 2025Feb 6, 2026Jun 8, 2026
Updated 4mo ago

Evaluation results

MethodLinks
ARZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.0651.25
Autoregressive TransformerZero-shot=true2026.0251.25
AR†2026.0651.25
Uni-EDLM-NCE2026.0658.14
EDLM-CoAR†2026.0660.23
Uni-EDLM-AR2026.0660.51
Block Diffusion*2026.0661.39
EDLM-NCE†2026.0663.19
DCD*2026.0666.51
MDLMZero-shot=true, Training Tokens=524B, Training Dataset=OWT2024.0667.01
MDLM†2026.0667.01
GaussianTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.0567.71
SEDDZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.0668.2
SEDD†2026.0668.2
Absorbing Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.0572.02
BaseTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.0572.06
Masked DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.0573.06
Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=22026.0273.8
DuoZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.0273.86
Duo‡2026.0673.86
Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=32026.0274.16
Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=52026.0274.91
UDLMZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.0277.59
Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.0580.33
Uniform DiffusionLoss=Cross-Entropy, Param.=LOO-Denoiser, Zero-shot transfer=true2026.0581.54
Uniform DiffusionLoss=ELBO, Param.=LOO-Denoiser, Zero-shot transfer=true2026.0582.03
SEDD UniformZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.0282.62
Uniform DiffusionLoss=Cross-Entropy, Param.=Denoiser, Zero-shot transfer=true2026.0583.43
Max Coupling Uniform DiffusionLoss=ELBO, Param.=LOO-Denoiser, Zero-shot transfer=true2026.05112.7
Max Coupling Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.05117.21