Recall-intensive Question Answering on SWDE, SQuAD, TQA, NQ, Drop (2K context, test)
56.1SWDEStateX
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| StateXArchitecture Family=State Space Model — Mamba2, Params=1.350B, Total State=37.44M2025.09 | 56.1 | 57.9 | 63.6 | 36.4 | 36.3 | 50.1 | |
| VanillaArchitecture Family=State Space Model — Mamba2, Params=1.343B, Total State=24.96M2025.09 | 54.1 | 57.8 | 63.5 | 36.8 | 35.4 | 49.5 | |
| StateXArchitecture Family=Linear Attention — GLA, Params=1.365B, Total State=18.87M2025.09 | 50.3 | 59.1 | 55 | 21.8 | 39.5 | 45.1 | |
| VanillaArchitecture Family=Linear Attention — GLA, Params=1.365B, Total State=12.58M2025.09 | 47.1 | 56.8 | 56 | 21.9 | 36.5 | 43.7 | |
| MoMArchitecture Family=Sparse Model – MoM, Params=1.552B, Total State=31.45M2025.09 | 34.4 | 49.6 | 50.1 | 16 | 33.9 | 36.8 |