List your company
ResearchFormerly WizWand

Language Modeling on WikiText-103 Medium (test)

35.37Perplexity (PPL)NAMEx-Full-Mom
35.164436.552237.9439.3278Oct 17, 2025
Updated 5mo ago

Evaluation results

MethodLinks
NAMEx-Full-MomParams=70M / 216M, Momentum=true2025.1035.37
SMoE (Top-2)Params=70M / 216M2025.1035.55
NAMEx-MomParams=70M / 216M, Momentum=true2025.1035.86
EP-CAMEx-MomParams=70M / 216M, Momentum=true2025.1036.16
NAMEx-FullParams=70M / 216M, Momentum=false2025.1036.21
NAMExParams=70M / 216M, Momentum=false2025.1036.4
CAMExParams=70M / 216M2025.1036.53
EP-CAMExParams=70M / 216M, Momentum=false2025.1036.55
SMEARParams=70M / 216M2025.1037.42
SMoE (Top-1)Params=70M / 216M2025.1040.51