Language Modeling on LM1B (val)
16.57PerplexityMDM-Prime-v2
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MDM-Prime-v2zero-shot=true, ℓ=16, variant=*2026.03 | 16.57 | — | |
| MDM-Prime-v2zero-shot=true, ℓ=162026.03 | 20.23 | — | |
| GPT2-BaseTokenizer=GPT-22025.10 | 22.32 | — | |
| Block Diffusion + Gumbel DistillationTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.03 | 22.69 | — | |
| TransformerCategory=Autoregressive, Retrained=true2026.04 | 22.8 | 66.7 | |
| Co-LADDLatent strategy=Joint, Loss weighting (λ_t^X)=1, Tokenizer=Qwen32025.10 | 24.49 | — | |
| CCDD-MMDiT w/ Qwen3Train. toks.=33B, # params.=216.2M2025.10 | 25.76 | — | |
| MDLM + Gumbel DistillationTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.03 | 26.01 | — | |
| MDLM†Total tokens seen=66B*2026.02 | 27.04 | — | |
| CCDD-MoEDiT w/ Qwen3Train. toks.=33B, # params.=104.0M2025.10 | 28.5 | — | |
| CCDD-MDiT w/ Qwen3Train. toks.=33B, # params.=92.1M2025.10 | 29.22 | — | |
| Co-LADDLatent strategy=Joint, Loss weighting (λ_t^X)=λ_t^{X,E}, Tokenizer=Qwen32025.10 | 29.99 | — | |
| LangFlowCategory=Diffusion (Uniform-state / Continuous)2026.04 | 30 | 92.2 | |
| MDLMCategory=Diffusion (Absorbing-state), Retrained=true2026.04 | 31 | 103.9 | |
| SEDD AbsorbCategory=Diffusion (Absorbing-state), Retrained=true2026.04 | 32 | 115.9 | |
| PlaidCategory=Diffusion (Uniform-state / Continuous), Retrained=true2026.04 | 32.4 | 77.3 | |
| SEDD AbsorbTokenizer=GPT-22025.10 | 32.79 | — | |
| MDLM2026.02 | 32.98 | — | |
| DuoCategory=Diffusion (Uniform-state / Continuous), Retrained=true2026.04 | 33.6 | 97.6 | |
| UDLMCategory=Diffusion (Uniform-state / Continuous), Retrained=true2026.04 | 33.8 | 99.8 | |
| MDLMLoss weighting (λ_t^X)=1, Tokenizer=Qwen32025.10 | 36.55 | — | |
| MDM-Primezero-shot=true, ℓ=82026.03 | 38 | — | |
| MDM-Primezero-shot=true, ℓ=42026.03 | 38.02 | — | |
| SCDDpu=0.12026.02 | 39.16 | — | |
| MDLM (reimpl.)Train. toks.=33B, # params.=92.1M2025.10 | 39.17 | — | |
| GIDD+pu=0.12026.02 | 39.98 | — | |
| MDLMLoss weighting (λ_t^X)=λ_t^{X,E}, Tokenizer=Qwen32025.10 | 39.99 | — | |
| SEDD UniformCategory=Diffusion (Uniform-state / Continuous)2026.04 | 40.3 | — | |
| GIDD+pu=0.22026.02 | 40.7 | — | |
| MDM-Primezero-shot=true, ℓ=6, variant=*2026.03 | 41.28 | — | |
| ARMzero-shot=true, variant=*2026.03 | 43.2 | — | |
| MDM-Primezero-shot=true, ℓ=62026.03 | 45.36 | — | |
| SCDDpu=0.22026.02 | 46.54 | — | |
| ARParam=0.1B, Evaluation Protocol=Zero-shot2026.05 | 51.14 | — | |
| ARMzero-shot=true2026.03 | 51.25 | — | |
| AR TransformerTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.06 | 52.14 | — | |
| MDM-Primezero-shot=true, ℓ=22026.03 | 55.5 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.1252025.06 | 60.11 | — | |
| EDLM-coARzero-shot=true2026.03 | 60.23 | — | |
| BD3-LMzero-shot=true2026.03 | 60.88 | — | |
| BDLMParam=0.1B, Training Tokens=256B, Evaluation Protocol=Zero-shot2026.05 | 60.88 | — | |
| MDMzero-shot=true, variant=*2026.03 | 60.96 | — | |
| Block DiffusionTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.03 | 61.14 | — | |
| EDLM-NCEzero-shot=true2026.03 | 63.19 | — | |
| DiffusionBertTokenizer=GPT-22025.10 | 63.78 | — | |
| DiffusionBertCategory=Diffusion (Absorbing-state)2026.04 | 63.8 | — | |
| MDLMParam=0.1B, Training Tokens=256B, Evaluation Protocol=Zero-shot2026.05 | 64.94 | — | |
| BD3-LMTraining steps=250K, Training Dataset=OWT, Zero-shot=true, L'=162025.06 | 65.11 | — | |
| MDLMTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.03 | 66.1 | — | |
| DCDMParam=0.1B, Training Tokens=128B, Evaluation Protocol=Zero-shot2026.05 | 66.1 | — | |
| MDMzero-shot=true2026.03 | 67.01 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.252025.06 | 67.69 | — | |
| LoopMDMS (inference-time loop count)=12, Zero-shot protocol=true, Model parameters=170M2026.05 | 67.8 | — | |
| LoopMDMS (inference-time loop count)=6, Zero-shot protocol=true, Model parameters=170M2026.05 | 68 | — | |
| MDMZero-shot protocol=true, Model parameters=170M2026.05 | 68.1 | — | |
| SEDDzero-shot=true2026.03 | 68.2 | — | |
| MDLMTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.06 | 70.79 | — | |
| SEDD AbsorbTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.06 | 72.51 | — | |
| Duozero-shot=true2026.03 | 73.86 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.52025.06 | 75.75 | — | |
| D3PM AbsorbCategory=Diffusion (Absorbing-state)2026.04 | 76.9 | — | |
| LoopMDMS (inference-time loop count)=1, Zero-shot protocol=true, Model parameters=170M2026.05 | 77.4 | — | |
| D3PMTokenizer=GPT-22025.10 | 77.5 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=12025.06 | 82.01 | — | |
| Diffusion-LMCategory=Diffusion (Uniform-state / Continuous)2026.04 | 118.6 | — | |
| Diffusion-LMTokenizer=GPT-22025.10 | 118.62 | — | |
| D3PM UniformCategory=Diffusion (Uniform-state / Continuous)2026.04 | 137.9 | — | |
| FLMCategory=Diffusion (Uniform-state / Continuous), Steps=10242026.04 | — | 96.9 |