List your company
ResearchFormerly WizWand

Language Modeling on WikiText-103 Medium (val)

33.76Perplexity (PPL)SMoE (Top-2)
33.566434.873236.1837.4868Oct 17, 2025
Updated 5mo ago

Evaluation results

MethodLinks
SMoE (Top-2)Params=70M / 216M2025.1033.76
NAMEx-Full-MomParams=70M / 216M, Momentum=true2025.1034.25
NAMEx-MomParams=70M / 216M, Momentum=true2025.1034.89
NAMEx-FullParams=70M / 216M, Momentum=false2025.1034.92
EP-CAMEx-MomParams=70M / 216M, Momentum=true2025.1035.09
NAMExParams=70M / 216M, Momentum=false2025.1035.14
CAMExParams=70M / 216M2025.1035.69
EP-CAMExParams=70M / 216M, Momentum=false2025.1035.78
SMEARParams=70M / 216M2025.1036.15
SMoE (Top-1)Params=70M / 216M2025.1038.6