List your company
ResearchFormerly WizWand

Language Modeling on text (val)

2.253Text NLL (Validation)SDM
2.235682.352592.46952.58641Jul 8, 2026
Updated 3mo ago

Evaluation results

MethodLinks
SDMScale=8B (L13), Local layer=SWA, Global layer=SDM2026.072.253
FullAttnScale=8B (L13), Local layer=SWA, Global layer=FullAttn2026.072.285
GDNScale=8B (L13), Local layer=SWA, Global layer=GDN2026.072.298
SDMScale=1.4B (L08), Local layer=SWA, Global layer=SDM2026.072.654
FullAttnScale=1.4B (L08), Local layer=SWA, Global layer=FullAttn2026.072.66
GDNScale=1.4B (L08), Local layer=SWA, Global layer=GDN2026.072.683
Mamba2Scale=1.4B (L08), Local layer=SWA, Global layer=Mamba22026.072.686