ResearchBenchmarksLanguage Modeling on Clustered Pre-training Data Cluster 9Language Modeling on Clustered Pre-training Data Cluster 9Follow15.405Perplexity (PPL)Split MoE15.3922415.4783715.564515.65063Mar 19, 2026Evaluation resultsMethodMethodLinksPerplexity (PPL)Split MoEModel Variant=Split MoEModel Variant=Split MoE2026.0315.405BaseModel Variant=BaseModel Variant=Base2026.0315.724