ResearchBenchmarksLanguage Modeling on WikiText-103 (train) - Training LossFollow2.422Training LossMiMuon2.32922.95563.5824.2084May 19, 2026Evaluation ResultsMethodMethodLinksTraining LossMiMuonModel=Qwen3-0.6BModel=Qwen3-0.6B2026.052.422MuSGDModel=Qwen3-0.6BModel=Qwen3-0.6B2026.052.834MuonModel=Qwen3-0.6BModel=Qwen3-0.6B2026.053.046ShampooModel=Qwen3-0.6BModel=Qwen3-0.6B2026.054.07LionModel=Qwen3-0.6BModel=Qwen3-0.6B2026.054.399AdamWModel=Qwen3-0.6BModel=Qwen3-0.6B2026.054.742