ResearchBenchmarksLanguage Modeling on Wikipedia out-distributionLanguage Modeling on Wikipedia out-distributionFollow22.07PerplexitySSA21.969622.647323.32524.0027Aug 20, 2025Evaluation resultsMethodMethodLinksPerplexitySSAModel Architecture=Nem...Model Architecture=Nemotron-style decoder, Parameters=114M, Training Steps=22K, Training Dataset=FineWeb2025.0822.07SoftmaxModel Architecture=Nem...Model Architecture=Nemotron-style decoder, Parameters=114M, Training Steps=22K, Training Dataset=FineWeb2025.0824.58