Knowledge-style Retrieval on SWDE 2048 tokens
53.1AccuracyLooped Hybrid (Full+GDN)
Evaluation results
| Method | Links | |
|---|---|---|
| Looped Hybrid (Full+GDN)Model (1.5B)=1.5B, Looped=true, Iterations=T=4, Mixer=Hybrid, Hybrid Ratio=1:12026.05 | 53.1 | |
| Looped TransformerModel (1.5B)=1.5B, Looped=true, Iterations=T=4, Mixer=Pure2026.05 | 52.8 | |
| Looped Hybrid (GDN+DSA)Model (1.5B)=1.5B, Looped=true, Iterations=T=4, Mixer=Hybrid, Hybrid Ratio=1:12026.05 | 51.6 | |
| TransformerModel (1.5B)=1.5B, Looped=false, Mixer=Pure2026.05 | 48.9 | |
| Looped GDNModel (1.5B)=1.5B, Looped=true, Iterations=T=4, Mixer=Pure2026.05 | 34.9 | |
| Looped Mamba-2Model (1.5B)=1.5B, Looped=true, Iterations=T=4, Mixer=Pure2026.05 | 33.9 | |
| GDNModel (1.5B)=1.5B, Looped=false, Mixer=Pure2026.05 | 32.7 | |
| Mamba-2Model (1.5B)=1.5B, Looped=false, Mixer=Pure2026.05 | 30.7 |