Question Answering on TriviaQA (TQA) 2048 tokens
68.2Accuracy (TQA)Looped Transformer
Evaluation results
| Method | Links | |
|---|---|---|
| Looped TransformerModel (1.5B)=1.5B, Looped=true, Iterations=T=42026.05 | 68.2 | |
| Looped Hybrid (Full+GDN)Model (1.5B)=1.5B, Looped=true, Iterations=T=42026.05 | 67.8 | |
| TransformerModel (1.5B)=1.5B2026.05 | 67.5 | |
| Looped Hybrid (GDN+DSA)Model (1.5B)=1.5B, Looped=true, Iterations=T=42026.05 | 66.9 | |
| Looped Mamba-2Model (1.5B)=1.5B, Looped=true, Iterations=T=42026.05 | 65.1 | |
| Looped GDNModel (1.5B)=1.5B, Looped=true, Iterations=T=42026.05 | 64.7 | |
| Mamba-2Model (1.5B)=1.5B2026.05 | 64.3 | |
| GDNModel (1.5B)=1.5B2026.05 | 63.5 |