ResearchBenchmarksLanguage Modeling on Mini-GPT-OSS (val)Language Modeling on Mini-GPT-OSS (val)Follow2.94Validation LossGrouter2.923763.033383.1433.25262Feb 22, 2026Evaluation resultsMethodMethodLinksValidation LossGrouterAttn=GQA, Structure=Mo...Attn=GQA, Structure=MoE, Tokens=30B2026.022.94ZLAttn=GQA, Structure=Mo...Attn=GQA, Structure=MoE, Tokens=30B2026.023.346