List your company
ResearchFormerly WizWand

Language Modeling on EnWik-8 (test)

1.122BPCSmoothSMoE
1.120761.129131.13751.14587Feb 21, 2025May 12, 2025Jul 31, 2025Oct 19, 2025Jan 7, 2026Mar 28, 2026Jun 17, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SmoothSMoE2026.061.122
Switch-ACMoEPre-training dataset=EnWik-82025.021.137
Switch TransformerPre-training dataset=EnWik-82025.021.153
SMoE2026.061.153