Language Modeling on OWT (val)
17.5PPLTransformer
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TransformerCategory=Autoregressive2026.04 | 17.5 | 35.9 | |
| LSSVariant=mHC, Model Size=L2026.06 | 22.78 | — | |
| MDLMCategory=Diffusion (Absorbing-state)2026.04 | 23.2 | 104.9 | |
| SEDD AbsorbCategory=Diffusion (Absorbing-state)2026.04 | 24.1 | — | |
| LSSVariant=mHC-lite, Model Size=L2026.06 | 24.29 | — | |
| LSS + IdentityVariant=mHC, Model Size=L2026.06 | 24.35 | — | |
| LSS + IdentityVariant=mHC-lite, Model Size=L2026.06 | 24.36 | — | |
| IdentityVariant=mHC, Model Size=L2026.06 | 24.45 | — | |
| BaselineVariant=mHC-lite, Model Size=L2026.06 | 24.51 | — | |
| IdentityVariant=mHC-lite, Model Size=L2026.06 | 24.54 | — | |
| LangFlowCategory=Diffusion (Uniform-state / Continuous)2026.04 | 24.6 | 36.5 | |
| BaselineVariant=mHC, Model Size=L2026.06 | 24.61 | — | |
| DuoCategory=Diffusion (Uniform-state / Continuous)2026.04 | 25.2 | 77.6 | |
| LSSVariant=mHC-lite, Model Size=M2026.06 | 25.65 | — | |
| LSS + IdentityVariant=mHC, Model Size=M2026.06 | 25.74 | — | |
| LSS + IdentityVariant=mHC-lite, Model Size=M2026.06 | 25.88 | — | |
| IdentityVariant=mHC-lite, Model Size=M2026.06 | 25.93 | — | |
| LSSVariant=mHC, Model Size=M2026.06 | 26.08 | — | |
| BaselineVariant=mHC-lite, Model Size=M2026.06 | 26.22 | — | |
| BaselineVariant=mHC, Model Size=M2026.06 | 26.44 | — | |
| IdentityVariant=mHC, Model Size=M2026.06 | 26.72 | — | |
| UDLMCategory=Diffusion (Uniform-state / Continuous)2026.04 | 27.4 | — | |
| SEDD UniformCategory=Diffusion (Uniform-state / Continuous)2026.04 | 29.7 | 103.6 | |
| FLMCategory=Diffusion (Uniform-state / Continuous), Steps=10242026.04 | — | 62.2 |