ResearchDatasetsPre-trainingPre-trainingFollowBenchmarksTaskDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyTaskDatasetSOTA resultTrendResultsLanguage ModelingPre-training (val)1.602Validation Loss64Pre-trainingPre-training (evaluation)3.254Pre-training Eval Loss5Pre-training efficiencyPre-training4,228Muon Steps4