ResearchBenchmarksLanguage Modeling on WikiText (Accuracy and LIS)Language Modeling on WikiText (Accuracy and LIS)Follow17.2826AccuracyRaw17.2066817.2263917.246117.26581Feb 23, 2026Evaluation resultsMethodMethodLinksAccuracyLISRawModel architecture=Swi...Model architecture=Switch Transformer (8 experts), Strategy=Unmodified model2026.0217.28262.09R&QModel architecture=Swi...Model architecture=Switch Transformer (8 experts), Strategy=Replicate-and-Quantize2026.0217.20961.6235