ResearchBenchmarksLanguage Modeling on Clustered Pre-training Data Cluster 8Language Modeling on Clustered Pre-training Data Cluster 8Follow14.96Perplexity (PPL)Split MoE14.9565614.9797815.00315.02622Mar 19, 2026Evaluation resultsMethodMethodLinksPerplexity (PPL)Split MoEModel Variant=Split MoEModel Variant=Split MoE2026.0314.96BaseModel Variant=BaseModel Variant=Base2026.0315.046