ResearchBenchmarksLanguage Modeling on LLaMA-60M (val)Follow3.37Validation LossNora3.36723.38613.4053.4239May 5, 2026Evaluation ResultsMethodMethodLinksValidation LossValidation PerplexityNoraWeight decay=0.0, Best...Weight decay=0.0, Best matrix LR=0.0042026.053.3728.94ManoWeight decay=0.1, Best...Weight decay=0.1, Best matrix LR=0.0042026.053.3929.55RMNPWeight decay=0.1, Best...Weight decay=0.1, Best matrix LR=0.0042026.053.4130.12MuonWeight decay=0.1, Best...Weight decay=0.1, Best matrix LR=0.012026.053.4431.09