ResearchBenchmarksPre-training on Pre-training (evaluation)Pre-training on Pre-training (evaluation)Follow3.254Pre-training Eval LossBHyT3.229963.392233.55453.71677Dec 26, 2025Evaluation resultsMethodMethodLinksPre-training Eval LossBHyTBackbone=Llama-3BBackbone=Llama-3B2025.123.254LNSBackbone=Llama-3BBackbone=Llama-3B2025.123.271RMSNormBackbone=Llama-3BBackbone=Llama-3B2025.123.272Peri-LNBackbone=Llama-3BBackbone=Llama-3B2025.123.279DyTBackbone=Llama-3BBackbone=Llama-3B2025.123.855