ResearchBenchmarksLanguage Modeling on DS Mini Lite V2 (val)Language Modeling on DS Mini Lite V2 (val)Follow2.469Validation LossGrouter2.445482.604242.7632.92176Feb 22, 2026Evaluation resultsMethodMethodLinksValidation LossGrouterAttn=MLA, Structure=De...Attn=MLA, Structure=Dense+MoE, Tokens=30B2026.022.469ZLAttn=MLA, Structure=De...Attn=MLA, Structure=Dense+MoE, Tokens=30B2026.023.057