ResearchBenchmarksLanguage Modeling on LLaMA-135M (val)Follow3.079Validation LossNora3.076483.093493.11053.12751May 5, 2026Evaluation ResultsMethodMethodLinksValidation LossValidation PerplexityNoraWeight decay=0.0, Best...Weight decay=0.0, Best matrix LR=0.0032026.053.07921.74ManoWeight decay=0.1, Best...Weight decay=0.1, Best matrix LR=0.0032026.053.09722.13RMNPWeight decay=0.1, Best...Weight decay=0.1, Best matrix LR=0.012026.053.11222.46MuonWeight decay=0.1, Best...Weight decay=0.1, Best matrix LR=0.012026.053.14223.17