Average Performance across Context Lengths on Long-context benchmarks
45.9Performance (8k Context)L2A
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| L2ABackbone=Qwen 2.5 1.5B2026.03 | 45.9 | 40.7 | 36.5 | 34.2 | 29.6 | |
| CLPBackbone=Qwen 2.5 1.5B2026.03 | 45.7 | 41.4 | 37.6 | 35 | 29.8 | |
| S^2-attnBackbone=Qwen 2.5 1.5B2026.03 | 45.7 | 41.6 | 36.2 | 30.7 | 22.8 | |
| BaseBackbone=Qwen 2.5 1.5B2026.03 | 42.3 | 39.4 | 34.2 | 23.2 | 12.8 | |
| DCABackbone=Qwen 2.5 1.5B2026.03 | 42.2 | 39.3 | 34.1 | 26.9 | 22.7 | |
| SelfExtendBackbone=Qwen 2.5 1.5B2026.03 | 42 | 39.4 | 32.4 | 26.8 | 22.4 | |
| NSABackbone=Qwen 2.5 1.5B2026.03 | 41.7 | 29.7 | 23.2 | 19.7 | 17 | |
| FlexPrefillBackbone=Qwen 2.5 1.5B2026.03 | 40 | 36.4 | 33.5 | 31 | 26.1 |