Language Modeling on RefinedWeb (val) benchmark leaderboard · SOTA2 Research

Language Modeling on RefinedWeb (val)

18.398PerplexityMLRA

Evaluation Results

Method	Links
MLRA2026.03		18.398
GLA2026.03		18.509
MLA2026.03		18.523
GLA2026.03		18.578
MLRA2026.03		18.602
MHA2026.03		18.698
GQA2026.03		18.885
GTA2026.03		18.924
TPA2026.03		18.971
MFA2026.03		19.125
MQA2026.03		19.202