List your company
ResearchFormerly WizWand

Language Modeling on Pubmed zero-shot

41.35PerplexityUni-EDLM-AR
40.371246.978153.58560.1919Jun 11, 2024Oct 10, 2024Feb 8, 2025Jun 9, 2025Oct 8, 2025Feb 6, 2026Jun 8, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Uni-EDLM-AR2026.0641.35
GaussianTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.0541.43
EDLM-NCE†2026.0641.8
MDLMZero-shot=true, Training Tokens=524B, Training Dataset=OWT2024.0641.89
MDLM†2026.0641.89
Block Diffusion*2026.0642.18
Uni-EDLM-NCE2026.0642.74
Absorbing Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.0543.01
Masked DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.0543.13
BaseTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.0543.15
Uniform DiffusionLoss=Cross-Entropy, Param.=LOO-Denoiser, Zero-shot transfer=true2026.0543.72
Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=22026.0243.98
DuoZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.0244.48
Duo‡2026.0644.48
Uniform DiffusionLoss=ELBO, Param.=LOO-Denoiser, Zero-shot transfer=true2026.0544.52
SEDDZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.0644.53
SEDD†2026.0644.53
Uniform DiffusionLoss=Cross-Entropy, Param.=Denoiser, Zero-shot transfer=true2026.0544.66
Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=32026.0244.87
Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.0545.75
EDLM-CoAR†2026.0646.31
Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=52026.0246.79
DCD*2026.0647.65
ARZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.0649.01
Autoregressive TransformerZero-shot=true2026.0249.01
AR†2026.0649.01
UDLMZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.0250.98
SEDD UniformZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.0255.89
Max Coupling Uniform DiffusionLoss=ELBO, Param.=LOO-Denoiser, Zero-shot transfer=true2026.0561.2
Max Coupling Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.0565.82