ResearchBenchmarksLanguage Modeling on 1024-token context (val)Language Modeling on 1024-token context (val)Follow28.56Best Perplexity (PPL)Full28.230830.452932.67534.8971Jun 1, 2026Evaluation resultsMethodMethodLinksBest Perplexity (PPL)FullArchitecture=FullArchitecture=Full2026.0628.56SWAArchitecture=SWAArchitecture=SWA2026.0630.56SE-BridgeArchitecture=SE-BridgeArchitecture=SE-Bridge2026.0630.84PBBArchitecture=PBBArchitecture=PBB2026.0631.38BridgeArchitecture=BridgeArchitecture=Bridge2026.0631.74BlockArchitecture=BlockArchitecture=Block2026.0636.79