Language Modeling on Wikitext zero-shot
24.01PerplexityUni-EDLM-NCE
Evaluation results
| Method | Links | |
|---|---|---|
| Uni-EDLM-NCE2026.06 | 24.01 | |
| ARZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.06 | 25.75 | |
| Autoregressive TransformerZero-shot=true2026.02 | 25.75 | |
| AR†2026.06 | 25.75 | |
| Uni-EDLM-AR2026.06 | 26.62 | |
| EDLM-CoAR†2026.06 | 28.31 | |
| EDLM-NCE†2026.06 | 30.77 | |
| Block Diffusion*2026.06 | 31.82 | |
| MDLMZero-shot=true, Training Tokens=524B, Training Dataset=OWT2024.06 | 32.83 | |
| MDLM†2026.06 | 32.83 | |
| DuoZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 33.57 | |
| Duo‡2026.06 | 33.57 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=22026.02 | 34.05 | |
| SEDDZero-shot=true, Training Tokens=524B, Training Dataset=OWT, Status=Retrained2024.06 | 34.28 | |
| SEDD†2026.06 | 34.28 | |
| GaussianTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.05 | 34.51 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=52026.02 | 34.52 | |
| Duo++Zero-shot=true, Number of Parameters=138M, Model Type=Diffusion, k=32026.02 | 34.65 | |
| DCD*2026.06 | 35.2 | |
| BaseTraining steps=1M, Training dataset=OpenWebText, Evaluation protocol=zero-shot2026.05 | 37.76 | |
| UDLMZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 39.42 | |
| SEDD UniformZero-shot=true, Number of Parameters=138M, Model Type=Diffusion2026.02 | 41.1 |