ResearchBenchmarksLanguage Modeling on Clustered Pre-training Data Cluster 11Language Modeling on Clustered Pre-training Data Cluster 11Follow22.002Perplexity (PPL)Split MoE21.9898822.0716922.153522.23531Mar 19, 2026Evaluation resultsMethodMethodLinksPerplexity (PPL)Split MoEModel Variant=Split MoEModel Variant=Split MoE2026.0322.002BaseModel Variant=BaseModel Variant=Base2026.0322.305