Language Understanding on LAMBADA 8 reasoning tasks
55.61AccuracyCARVE + SWA
Evaluation results
| Method | Links | |
|---|---|---|
| CARVE + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 55.61 | |
| GDN-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 53.97 | |
| CARVEModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 53.74 | |
| GDN-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 53.11 | |
| Mamba-3 MIMO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 52.72 | |
| Mamba-3 SISO + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 52.69 | |
| KDA + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 52.68 | |
| Mamba-3 MIMOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 52.39 | |
| KDAModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 52.28 | |
| Gated DeltaNet + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 52.25 | |
| Gated DeltaNetModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 52.07 | |
| Mamba-2 + SWAModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 51.99 | |
| Mamba-2Model Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 51.82 | |
| Mamba-3 SISOModel Category=Recurrent, Parameter Scale=1.3B, Token Scale=100B2026.06 | 51.42 | |
| TransformerModel Category=Hybrid, Parameter Scale=1.3B, Token Scale=100B2026.06 | 50.86 |