Long-context understanding on LongBench (Key Subtask Metrics)
25GvR ScoreSwiAttn
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SwiAttn2026.03 | 25 | 21.3 | 35.3 | 18 | 69.9 | 32.4 | 60 | 37.4 | |
| FullAttn2026.03 | 24.9 | 20.9 | 33.9 | 18.6 | 69.3 | 30.2 | 63.7 | 37.4 | |
| StaticHybrid2026.03 | 22.6 | 20 | 29.6 | 14.5 | 68.1 | 32.3 | 59.3 | 35.2 | |
| SWA-ZSTraining Strategy=Zero-shot2026.03 | 19.2 | 18.2 | 18.5 | 10.5 | 51.1 | 27.4 | 18.7 | 23.4 | |
| SWA-CPTTraining Strategy=Continued Pre-training2026.03 | 19.2 | 18.7 | 24.2 | 13.1 | 57.8 | 30.7 | 53 | 31 |