List your company
ResearchFormerly WizWand

Language Modeling on WikiText (val)

12.51PerplexityMDM-Prime-v2
9.680428.780247.8866.9798Jun 2, 2025Jul 31, 2025Sep 29, 2025Nov 27, 2025Jan 26, 2026Mar 26, 2026May 25, 2026
Updated 5mo ago

Evaluation results

MethodLinks
MDM-Prime-v2zero-shot=true, ℓ=16, variant=*2026.0312.51
Block Diffusion + Gumbel DistillationTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.0313.86
MDLM + Gumbel DistillationTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.0315.57
MDM-Primezero-shot=true, ℓ=6, variant=*2026.0319.46
AsyncMeshMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.0121.14
FullSyncMesh Configuration=4 x 2, Iterations=30k, AsyncPP=false2026.0121.23
DPAvgMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.0121.26
SPARTAMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.0121.3
ARMzero-shot=true, variant=*2026.0322.28
MDM-Primezero-shot=true, ℓ=42026.0323.86
AsyncSPARTAMesh Configuration=4 x 2, Iterations=30k, AsyncPP=true2026.0124.8
ARParam=0.1B, Evaluation Protocol=Zero-shot2026.0525.32
ARMzero-shot=true2026.0325.75
MDM-Primezero-shot=true, ℓ=82026.0325.77
MDM-Prime-v2zero-shot=true, ℓ=162026.0326.05
AR TransformerTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.0626.54
MDMzero-shot=true, variant=*2026.0326.65
MDM-Primezero-shot=true, ℓ=62026.0326.87
MDM-Primezero-shot=true, ℓ=22026.0327.93
EDLM-coARzero-shot=true2026.0328.31
EDLM-NCEzero-shot=true2026.0330.77
Block DiffusionTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.0330.82
MASA-QKVModel size=729M, Attn CR=50.0%, Zero-shot=true2025.0830.83
Transformer-LModel size=729M, Attn CR=0%, Zero-shot=true2025.0830.88
BD3-LMzero-shot=true2026.0331.31
BDLMParam=0.1B, Training Tokens=256B, Evaluation Protocol=Zero-shot2026.0531.31
MASA-QKVOModel size=729M, Attn CR=66.7%, Zero-shot=true2025.0831.34
GQAModel size=729M, Attn CR=41.7%, Zero-shot=true2025.0831.74
DCDMParam=0.1B, Training Tokens=128B, Evaluation Protocol=Zero-shot2026.0531.94
Repeat-all-overModel size=729M, Attn CR=66.7%, Zero-shot=true2025.0832.27
LoopMDMS (inference-time loop count)=12, Zero-shot protocol=true, Model parameters=170M2026.0532.4
Seq-SharingModel size=729M, Attn CR=66.7%, Zero-shot=true2025.0832.43
MDMzero-shot=true2026.0332.83
BD3-LMTraining steps=250K, Training Dataset=OWT, Zero-shot=true, L'=162025.0632.88
MDLMParam=0.1B, Training Tokens=256B, Evaluation Protocol=Zero-shot2026.0533.22
Low-RankModel size=729M, Attn CR=66.7%, Zero-shot=true2025.0833.28
Duozero-shot=true2026.0333.57
LoopMDMS (inference-time loop count)=6, Zero-shot protocol=true, Model parameters=170M2026.0534.2
SEDDzero-shot=true2026.0334.28
MDLMTraining Dataset=OWT, Evaluation Protocol=Zero-shot2026.0334.52
MDMZero-shot protocol=true, Model parameters=170M2026.0535.1
Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.1252025.0635.65
MDLMTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.0637.08
Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.252025.0637.32
SEDD AbsorbTraining steps=250K, Training Dataset=OWT, Zero-shot=true2025.0638.55
Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=0.52025.0639.57
LoopMDMS (inference-time loop count)=1, Zero-shot protocol=true, Model parameters=170M2026.0541.8
MASA-QKVModel size=335M, Attn CR=50.0%, Zero-shot=true2025.0842.31
Transformer-MModel size=335M, Attn CR=0%, Zero-shot=true2025.0844.49
MASA-QKVOModel size=335M, Attn CR=66.7%, Zero-shot=true2025.0845
Eso-LMsTraining steps=250K, Training Dataset=OWT, Zero-shot=true, alpha_0=12025.0645.08
GQAModel size=335M, Attn CR=43.8%, Zero-shot=true2025.0846.21
Seq-SharingModel size=335M, Attn CR=66.7%, Zero-shot=true2025.0847.36
Low-RankModel size=335M, Attn CR=66.7%, Zero-shot=true2025.0847.48
Repeat-all-overModel size=335M, Attn CR=66.7%, Zero-shot=true2025.0847.63
MASA-QKVModel size=110M, Attn CR=50.0%, Zero-shot=true2025.0872.08
MASA-QKVOModel size=110M, Attn CR=66.7%, Zero-shot=true2025.0872.82
Transformer-SModel size=110M, Attn CR=0%, Zero-shot=true2025.0876.11
GQAModel size=110M, Attn CR=41.7%, Zero-shot=true2025.0878.41
Repeat-all-overModel size=110M, Attn CR=66.7%, Zero-shot=true2025.0878.97
Seq-SharingModel size=110M, Attn CR=66.7%, Zero-shot=true2025.0880.35
Low-RankModel size=110M, Attn CR=66.7%, Zero-shot=true2025.0883.25