Long-context language understanding on LongBench (standard)
5.89NQATraining–Inference Consistent Segmented Execution
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Training–Inference Consistent Segmented ExecutionBackbone=LLaMA2-7B-80K2026.05 | 5.89 | 8.22 | 8.11 | 9.68 | 6.85 | 9.87 | 3.81 | 12.77 | 9.38 | 16.81 | 10.3 | 63.5 | 88.22 | 41.99 | 58.47 | 51.93 | 25.36 | |
| NSABackbone=LLaMA2-7B-80K2026.05 | 2.24 | 5.23 | 7.1 | 8.34 | 5.44 | 6.99 | 3.67 | 11.68 | 10.1 | 15.14 | 4.52 | 60 | 73.27 | 26.84 | 53.94 | 44.11 | 21.16 | |
| LLaMA2-7B-80KBackbone=LLaMA2-7B-80K2026.05 | 2.07 | 4.99 | 6.64 | 3.59 | 6 | 4.15 | 2.32 | 17.97 | 10.55 | 20.34 | 8.56 | 69 | 86.75 | 43.29 | 55.66 | 45.33 | 24.2 | |
| MInferenceBackbone=LLaMA2-7B-80K2026.05 | 1.92 | 5.04 | 6.44 | 3.58 | 5.6 | 3.87 | 2.66 | 18.38 | 10.32 | 20.85 | 8.81 | 69.5 | 86.67 | 43.74 | 55.82 | 45.3 | 24.28 | |
| DuoAttentionBackbone=LLaMA2-7B-80K2026.05 | 1.56 | 5.28 | 5.81 | 4.61 | 5.79 | 4.25 | 2.74 | 16.95 | 11.26 | 19.37 | 9.73 | 69 | 87.12 | 31.36 | 56.07 | 45.58 | 23.53 | |
| StreamingLLMBackbone=LLaMA2-7B-80K2026.05 | 1.03 | 3.06 | 4.93 | 1.88 | 6.83 | 2.31 | 2.81 | 8.1 | 5.03 | 15.3 | 12.04 | 60.5 | 87.07 | 41.36 | 57.32 | 45.86 | 22.21 |