Language Modeling on Arxiv zero-shot
35.53PerplexityUni-EDLM-NCE
Evaluation results
| Method | Links | |
|---|---|---|
| Uni-EDLM-NCE2026.06 | 35.53 | |
| EDLM-NCE†2026.06 | 36.63 | |
| MDLMZero-shot=true, Training Tokens=524B, Training Dataset=OWT2024.06 | 37.37 | |
| MDLM†2026.06 | 37.37 | |
| Uni-EDLM-AR2026.06 | 37.37 | |
| BaseTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.05 | 37.85 | |
| GaussianTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.05 | 37.88 | |
| SEDDZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.06 | 38.48 | |
| SEDD†2026.06 | 38.48 | |
| Block Diffusion*2026.06 | 38.74 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=22026.02 | 38.93 | |
| EDLM-CoAR†2026.06 | 39.02 | |
| DCD*2026.06 | 40.09 | |
| DuoZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 40.39 | |
| Duo‡2026.06 | 40.39 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=32026.02 | 40.42 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=52026.02 | 41.04 | |
| ARZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.06 | 41.73 | |
| Autoregressive TransformerZero-shot=true2026.02 | 41.73 | |
| AR†2026.06 | 41.73 | |
| UDLMZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 44.08 | |
| SEDD UniformZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 50.86 |