Language Modeling on Pubmed zero-shot
41.35PerplexityUni-EDLM-AR
Evaluation Results
| Method | Links | |
|---|---|---|
| Uni-EDLM-AR2026.06 | 41.35 | |
| GaussianTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.05 | 41.43 | |
| EDLM-NCE†2026.06 | 41.8 | |
| MDLMZero-shot=true, Training Tokens=524B, Training Dataset=OWT2024.06 | 41.89 | |
| MDLM†2026.06 | 41.89 | |
| Block Diffusion*2026.06 | 42.18 | |
| Uni-EDLM-NCE2026.06 | 42.74 | |
| Absorbing Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.05 | 43.01 | |
| Masked DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.05 | 43.13 | |
| BaseTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.05 | 43.15 | |
| Uniform DiffusionLoss=Cross-Entropy, Param.=LOO-Denoiser, Zero-shot transfer=true2026.05 | 43.72 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=22026.02 | 43.98 | |
| DuoZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 44.48 | |
| Duo‡2026.06 | 44.48 | |
| Uniform DiffusionLoss=ELBO, Param.=LOO-Denoiser, Zero-shot transfer=true2026.05 | 44.52 | |
| SEDDZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.06 | 44.53 | |
| SEDD†2026.06 | 44.53 | |
| Uniform DiffusionLoss=Cross-Entropy, Param.=Denoiser, Zero-shot transfer=true2026.05 | 44.66 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=32026.02 | 44.87 | |
| Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.05 | 45.75 | |
| EDLM-CoAR†2026.06 | 46.31 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=52026.02 | 46.79 | |
| DCD*2026.06 | 47.65 | |
| ARZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.06 | 49.01 | |
| Autoregressive TransformerZero-shot=true2026.02 | 49.01 | |
| AR†2026.06 | 49.01 | |
| UDLMZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 50.98 | |
| SEDD UniformZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 55.89 | |
| Max Coupling Uniform DiffusionLoss=ELBO, Param.=LOO-Denoiser, Zero-shot transfer=true2026.05 | 61.2 | |
| Max Coupling Uniform DiffusionLoss=ELBO, Param.=Denoiser, Zero-shot transfer=true2026.05 | 65.82 |