List your company
ResearchFormerly WizWand

Language Modeling on WikiText-103 Attacked (test)

41.351Perplexity (PPL)SSR-L
41.228842.0536542.878543.70335Nov 12, 2025Dec 18, 2025Jan 23, 2026Feb 28, 2026Apr 5, 2026May 11, 2026Jun 17, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SSR-LSetting=Momentum setting, Training Time Overhead=0.36%2025.1141.351
SSR-L w/ noiseSetting=Momentum setting, Training Time Overhead=1.58%2025.1141.627
SSR-S w/ noiseSetting=Momentum setting, Training Time Overhead=1.76%2025.1141.639
Vanilla SMoESetting=Momentum setting, Training Time Overhead=0.00%2025.1141.756
SSR-SSetting=Momentum setting, Training Time Overhead=0.61%2025.1142.104
SMoE w/ noiseSetting=Momentum setting, Training Time Overhead=6.75%2025.1142.204
SMoE w/ z-lossSetting=Momentum setting, Training Time Overhead=4.44%2025.1142.614
Sinkhorn-based SMoESetting=Momentum setting, Training Time Overhead=67.52%2025.1142.803
SmoothSMoETotal Experts=16, Routing=top-22026.0642.85
SSR-S w/ noiseSetting=Conventional setting, Training Time Overhead=0.65%2025.1142.941
SMoE w/ lb lossSetting=Momentum setting, Training Time Overhead=2.49%2025.1142.998
SSR-L w/ noiseSetting=Conventional setting, Training Time Overhead=0.62%2025.1143.159
SSR-SSetting=Conventional setting, Training Time Overhead=0.37%2025.1143.283
SSR-LSetting=Conventional setting, Training Time Overhead=0.33%2025.1143.318
SMoE w/ noiseSetting=Conventional setting, Training Time Overhead=2.50%2025.1143.692
SMoE w/ lb lossSetting=Conventional setting, Training Time Overhead=2.04 %2025.1143.758
ReMoETotal Experts=16, Routing=top-22026.0644
SMoETotal Experts=16, Routing=top-22026.0644.18
Vanilla SMoESetting=Conventional setting, Training Time Overhead=0.00%2025.1144.19
Sinkhorn-based SMoESetting=Conventional setting, Training Time Overhead=72.47%2025.1144.351
SMoE w/ z-lossSetting=Conventional setting, Training Time Overhead=2.12%2025.1144.406