Question Answering on Natural Questions (NQ) (2048 tokens context)
34AccuracyLooped Hybrid (Full+GDN)
Evaluation results
| Method | Links | |
|---|---|---|
| Looped Hybrid (Full+GDN)Model (1.5B)=1.5B, Looped=true2026.05 | 34 | |
| Looped TransformerModel (1.5B)=1.5B, Looped=true2026.05 | 33.6 | |
| Looped Hybrid (GDN+DSA)Model (1.5B)=1.5B, Looped=true2026.05 | 33 | |
| TransformerModel (1.5B)=1.5B2026.05 | 31.7 | |
| Looped GDNModel (1.5B)=1.5B, Looped=true2026.05 | 27 | |
| Looped Mamba-2Model (1.5B)=1.5B, Looped=true2026.05 | 26.8 | |
| GDNModel (1.5B)=1.5B2026.05 | 25.7 | |
| Mamba-2Model (1.5B)=1.5B2026.05 | 25.1 |