ResearchBenchmarksPretraining on Llama pretraining 124MFor AI assistantsMarkdownContent indexPretraining on Llama pretraining 124MFollow0.674Iteration Time (s)AdamW0.67220.684350.69650.70865May 21, 2026Evaluation resultsMethodMethodLinksIteration Time (s)Relative TimeThroughput (tokens/s)AdamW2026.050.6741177,423SF-AdamW2026.050.6841.015175,213Muon2026.050.7061.047169,222AMUSE2026.050.7191.067166,697