ResearchBenchmarksLanguage Modeling on 20B token pretraining corpus (Train/Eval)Language Modeling on 20B token pretraining corpus (Train/Eval)Follow2.756PT Train LossBHyT2.75382.768652.78352.79835Dec 26, 2025Evaluation resultsMethodMethodLinksPT Train LossPT Eval LossBHyTPT Wall time=171.4h, M...PT Wall time=171.4h, Model=Llama-3B2025.122.7562.76Peri-LNPT Wall time=238h, Mod...PT Wall time=238h, Model=Llama-3B2025.122.8112.812