ResearchBenchmarksLanguage Modeling on Wikipedia (val)Language Modeling on Wikipedia (val)Follow14.407PerplexityMLRA14.3555614.7027815.0515.39722Mar 2, 2026Evaluation resultsMethodMethodLinksPerplexityMLRABranches=4Branches=42026.0314.407GLABranches=4Branches=42026.0314.547MLA2026.0314.567GLABranches=2Branches=22026.0314.605MLRABranches=2Branches=22026.0314.615MHA2026.0314.624GTA2026.0314.733TPA2026.0314.789GQA2026.0315.057MQA2026.0315.134MFA2026.0315.693