List your company
ResearchFormerly WizWand

Question Answering on OpenBookQA (acc)

40.6AccuracyU-NorMuon
16.05622.42828.835.172Jun 25, 2026Jun 26, 2026Jun 27, 2026Jun 28, 2026Jun 29, 2026Jun 30, 2026Jul 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
U-NorMuonModel Size=1.1B, Evaluation Protocol=0-shot2026.0640.6
NorMuonModel Size=1.1B, Evaluation Protocol=0-shot2026.0639.4
MuonModel Size=1.1B, Evaluation Protocol=0-shot2026.0639.2
AuroraModel Size=1.1B, Evaluation Protocol=0-shot2026.0639.2
AuroraModel Size=340M, Evaluation Protocol=0-shot2026.0635.6
MuonModel Size=340M, Evaluation Protocol=0-shot2026.0634.6
CARVE + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0634.2
NorMuonModel Size=340M, Evaluation Protocol=0-shot2026.0634.2
U-NorMuonModel Size=340M, Evaluation Protocol=0-shot2026.0633.4
GDN-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0633
CARVEModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0632.2
Mamba-3 SISO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0632
GDN-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0631.6
Mamba-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0631
Mamba-3 SISOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0631
Gated DeltaNet + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0630.6
KDAModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0630.4
Gated DeltaNetModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0630.2
Mamba-3 MIMOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.0630
KDA + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0630
Mamba-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0629.8
Mamba-3 MIMO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0629.4
TransformerModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.0625
SCAPESparsity (d)=0.1, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.0719.4
ADAMWSynchronization=DENSE all-reduce, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.0718.8
ADAMSSynchronization=DENSE all-reduce, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.0718.8
SCAPE (d = 0.01)Protocol=Zero-shot, Model=GPT-345M, d=0.012026.0718.6
SCAPESparsity (d)=0.01, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.0718.6
ADAMS (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.0718
SCAPE (d = 0.1)Protocol=Zero-shot, Model=GPT-345M, d=0.12026.0717.8
ADAMW (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.0717