Long-context Reasoning on LongBench (Detailed Metrics)
16.6NQANirvana
Evaluation results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| NirvanaModel Category=Attention or hybrid models2025.10 | 16.6 | 12.8 | 26 | 14.6 | 24.8 | 9.7 | 10.4 | 15.9 | 15.4 | 36.4 | 25.2 | 22.6 | 17.5 | 21.5 | 19.2 | |
| Nirvana-noTriggerModel Category=Attention or hybrid models2025.10 | 14.8 | 11.8 | 25.6 | 14 | 23.9 | 7.7 | 9.2 | 15.1 | 13.5 | 33 | 21.2 | 22.9 | 16.5 | 20.9 | 17.9 | |
| Gated DeltaNet-H1Model Category=Attention or hybrid models2025.10 | 14.5 | 12.3 | 26.6 | 12.6 | 23.6 | 6.1 | 9.1 | 16.1 | 12.8 | 33.5 | 23.9 | 26.8 | 15.5 | 19.2 | 17.8 | |
| Gated DeltaNetModel Category=Recurrent models2025.10 | 14.1 | 14 | 23.3 | 13.7 | 14.4 | 5.8 | 7.5 | 16.4 | 7.9 | 30 | 22.4 | 23 | 18.7 | 22.1 | 16.6 | |
| MambaModel Category=Recurrent models2025.10 | 13 | 10.1 | 20.4 | 10.1 | 16.7 | 6 | 7.2 | 15.9 | 8.4 | 23.1 | 21.9 | 11.2 | 17.9 | 19 | 14.6 | |
| DeltaNetModel Category=Recurrent models2025.10 | 12.9 | 10.8 | 21.5 | 10.9 | 13.2 | 5.1 | 6.5 | 13.5 | 7.2 | 15.5 | 23.3 | 11.6 | 17.6 | 20.3 | 13.6 | |
| Gated DeltaNet-H2Model Category=Attention or hybrid models2025.10 | 12.7 | 13 | 27.1 | 12.7 | 20.6 | 7.5 | 10.4 | 16.2 | 13 | 40.5 | 22.7 | 27.9 | 19.9 | 22.1 | 18.4 | |
| SambaModel Category=Attention or hybrid models2025.10 | 12.5 | 12.9 | 25.4 | 11.2 | 19.7 | 6.8 | 9.1 | 15.7 | 11 | 20 | 22.7 | 22.8 | 18.1 | 21.1 | 15.9 | |
| RetNetModel Category=Recurrent models2025.10 | 12.1 | 10.7 | 19.1 | 10.7 | 18 | 5.8 | 4.8 | 15.8 | 7.9 | 19 | 18 | 12.8 | 14.1 | 17.9 | 13.2 | |
| Transformer++Model Category=Attention or hybrid models2025.10 | 11.8 | 9.3 | 10 | 10.9 | 4.2 | 6.1 | 7.4 | 15.8 | 6.6 | 16.9 | 13.5 | 3.9 | 17.2 | 18.7 | 11 | |
| Mamba2Model Category=Recurrent models2025.10 | 11.1 | 11.3 | 18.6 | 11.8 | 15.1 | 6.7 | 6.7 | 14.5 | 7.4 | 13 | 23.6 | 8.4 | 17.9 | 20.6 | 13.5 | |
| HGRN2Model Category=Recurrent models2025.10 | 10.7 | 12.1 | 19.1 | 11.3 | 15.7 | 6 | 5.2 | 15.1 | 9.2 | 16 | 15.8 | 10.3 | 18.6 | 20.8 | 13.5 |