ResearchBenchmarksLanguage Modeling on Clustered Pre-training Data Cluster 6Language Modeling on Clustered Pre-training Data Cluster 6Follow15.097Perplexity (PPL)Split MoE15.0788415.2014215.32415.44658Mar 19, 2026Evaluation resultsMethodMethodLinksPerplexity (PPL)Split MoEModel Variant=Split MoEModel Variant=Split MoE2026.0315.097BaseModel Variant=BaseModel Variant=Base2026.0315.551