Long-context Retrieval on NIAH Single 3 (Accuracies at 1024, 2048, 4096)
100Accuracy (1024)Looped Hybrid (GDN+DSA)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Looped Hybrid (GDN+DSA)Model (1.5B)=1.5B, Looped=true2026.05 | 100 | 99.6 | 60.3 | |
| Looped Hybrid (Full+GDN)Model (1.5B)=1.5B, Looped=true2026.05 | 99.8 | 99.8 | 63.7 | |
| Looped TransformerModel (1.5B)=1.5B, Looped=true2026.05 | 99.2 | 99.8 | 0 | |
| TransformerModel (1.5B)=1.5B2026.05 | 98.6 | 99.4 | 0 | |
| Looped Mamba-2Model (1.5B)=1.5B, Looped=true2026.05 | 96.2 | 88.1 | 16.2 | |
| Mamba-2Model (1.5B)=1.5B2026.05 | 95.8 | 87.4 | 13.4 | |
| CCQ-GLAScale=1.3B2026.05 | 94.4 | 90.4 | 64.8 | |
| TransformerScale=1.3B2026.05 | 93 | 94.4 | 82.2 | |
| Looped GDNModel (1.5B)=1.5B, Looped=true2026.05 | 85.6 | 71 | 35.8 | |
| GDNModel (1.5B)=1.5B2026.05 | 83.8 | 68.4 | 34.2 | |
| Mamba2Scale=1.3B2026.05 | 69.4 | 47.6 | 27.4 | |
| Mamba2Scale=500M2026.05 | 68.6 | 54.8 | 28.8 | |
| CCQ-Gated DeltaNetScale=500M2026.05 | 67.6 | 53.4 | 33.6 | |
| CCQ-Gated DeltaNetScale=1.3B2026.05 | 66.4 | 59.2 | 46.6 | |
| GLAScale=1.3B2026.05 | 66 | 51.6 | 29.2 | |
| Gated DeltaNetScale=1.3B2026.05 | 64.4 | 56 | 6 | |
| Gated DeltaNetScale=500M2026.05 | 56.2 | 54.2 | 30.2 | |
| GLA-HedgehogScale=1.3B2026.05 | 49.2 | 23.8 | 1.8 | |
| GLAScale=500M2026.05 | 39.8 | 16 | 4.2 | |
| TransformerScale=500M2026.05 | 12.2 | 7.2 | 12.6 | |
| GLA-HedgehogScale=500M2026.05 | 11.8 | 3.4 | 0.4 | |
| CCQ-GLAScale=500M2026.05 | 11 | 41.4 | 3.6 |