Language Modeling on Arxiv (val)
18.22PerplexityMDM-Prime-v2
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MDM-Prime-v2zero-shot=true, ℓ=16, variant=*2026.03 | 18.22 | — | |
| CR-NetBackbone=LLaMA-2 1B, Pre-training Tokens=15.6B, Fine-tuning Epochs=82025.09 | 24.48 | 41 | |
| GaLoreBackbone=LLaMA-2 1B, Pre-training Tokens=15.6B, Fine-tuning Epochs=82025.09 | 25.17 | 40.59 | |
| MDM-Primezero-shot=true, ℓ=62026.03 | 25.19 | — | |
| MDM-Primezero-shot=true, ℓ=22026.03 | 25.44 | — | |
| MDM-Primezero-shot=true, ℓ=82026.03 | 25.79 | — | |
| MDM-Primezero-shot=true, ℓ=42026.03 | 25.83 | — | |
| MDM-Primezero-shot=true, ℓ=6, variant=*2026.03 | 26.24 | — | |
| ARMzero-shot=true, variant=*2026.03 | 29.82 | — | |
| MDM-Prime-v2zero-shot=true, ℓ=162026.03 | 31.28 | — | |
| MDMzero-shot=true, variant=*2026.03 | 31.89 | — | |
| LoopMDMS (inference-time loop count)=12, Zero-shot protocol=true, Model parameters=170M2026.05 | 32.7 | — | |
| LoopMDMS (inference-time loop count)=6, Zero-shot protocol=true, Model parameters=170M2026.05 | 33.1 | — | |
| DCDMParam=0.1B, Training Tokens=128B, Evaluation Protocol=Zero-shot2026.05 | 35.41 | — | |
| EDLM-NCEzero-shot=true2026.03 | 36.63 | — | |
| MDMzero-shot=true2026.03 | 37.37 | — | |
| MDMZero-shot protocol=true, Model parameters=170M2026.05 | 37.8 | — | |
| MDLMParam=0.1B, Training Tokens=256B, Evaluation Protocol=Zero-shot2026.05 | 37.89 | — | |
| SEDDzero-shot=true2026.03 | 38.48 | — | |
| EDLM-coARzero-shot=true2026.03 | 39.02 | — | |
| BD3-LMzero-shot=true2026.03 | 39.2 | — | |
| BDLMParam=0.1B, Training Tokens=256B, Evaluation Protocol=Zero-shot2026.05 | 39.2 | — | |
| BD3-LMTraining steps=250K, Training Dataset=OWT, Zero-shot=true, L'=162025.06 | 40.13 | — | |
| MDLMTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.06 | 40.21 | — | |
| Duozero-shot=true2026.03 | 40.39 | — | |
| SEDD AbsorbTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.06 | 41.18 | — | |
| ARParam=0.1B, Evaluation Protocol=Zero-shot2026.05 | 41.22 | — | |
| ARMzero-shot=true2026.03 | 41.73 | — | |
| LoopMDMS (inference-time loop count)=1, Zero-shot protocol=true, Model parameters=170M2026.05 | 42.5 | — | |
| AR TransformerTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.06 | 44.98 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.52025.06 | 53.78 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.252025.06 | 55.31 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=12025.06 | 55.76 | — | |
| Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.1252025.06 | 57.4 | — |