Question Answering on OpenBookQA (acc)
40.6AccuracyU-NorMuon
Evaluation Results
| Method | Links | |
|---|---|---|
| U-NorMuonModel Size=1.1B, Evaluation Protocol=0-shot2026.06 | 40.6 | |
| NorMuonModel Size=1.1B, Evaluation Protocol=0-shot2026.06 | 39.4 | |
| MuonModel Size=1.1B, Evaluation Protocol=0-shot2026.06 | 39.2 | |
| AuroraModel Size=1.1B, Evaluation Protocol=0-shot2026.06 | 39.2 | |
| AuroraModel Size=340M, Evaluation Protocol=0-shot2026.06 | 35.6 | |
| MuonModel Size=340M, Evaluation Protocol=0-shot2026.06 | 34.6 | |
| CARVE + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 34.2 | |
| NorMuonModel Size=340M, Evaluation Protocol=0-shot2026.06 | 34.2 | |
| U-NorMuonModel Size=340M, Evaluation Protocol=0-shot2026.06 | 33.4 | |
| GDN-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 33 | |
| CARVEModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 32.2 | |
| Mamba-3 SISO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 32 | |
| GDN-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 31.6 | |
| Mamba-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 31 | |
| Mamba-3 SISOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 31 | |
| Gated DeltaNet + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 30.6 | |
| KDAModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 30.4 | |
| Gated DeltaNetModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 30.2 | |
| Mamba-3 MIMOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 30 | |
| KDA + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 30 | |
| Mamba-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 29.8 | |
| Mamba-3 MIMO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 29.4 | |
| TransformerModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 25 | |
| SCAPESparsity (d)=0.1, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.07 | 19.4 | |
| ADAMWSynchronization=DENSE all-reduce, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.07 | 18.8 | |
| ADAMSSynchronization=DENSE all-reduce, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.07 | 18.8 | |
| SCAPE (d = 0.01)Protocol=Zero-shot, Model=GPT-345M, d=0.012026.07 | 18.6 | |
| SCAPESparsity (d)=0.01, Backbone=Llama-500M, Evaluation Protocol=Zero-shot2026.07 | 18.6 | |
| ADAMS (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.07 | 18 | |
| SCAPE (d = 0.1)Protocol=Zero-shot, Model=GPT-345M, d=0.12026.07 | 17.8 | |
| ADAMW (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.07 | 17 |