Long-context language understanding on LongBench (7-Metric Subset: NQ, HQ, QM, MN, GR, TQ, Avg)
29.52NQ ScoreDASH-KV
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| DASH-KVBackbone=Qwen2.5-14B-Instruct, Type=Retrieval2026.04 | 29.52 | 61.83 | 45.79 | 14.44 | 18.06 | 87.88 | 42.92 | |
| Full AttnBackbone=Qwen2.5-14B-Instruct, Type=Dense2026.04 | 28.2 | 61.98 | 45.52 | 14.19 | 16.78 | 86.91 | 42.26 | |
| Full AttnBackbone=Llama-3.1-8B-Instruct, Type=Dense2026.04 | 28.11 | 57.43 | 45.29 | 15.2 | 19.97 | 90.22 | 42.7 | |
| DASH-KVBackbone=Llama-3.1-8B-Instruct, Type=Retrieval2026.04 | 27.61 | 58.01 | 45.35 | 14.9 | 19.25 | 89.47 | 42.43 | |
| Full AttnBackbone=Qwen2-7B-Instruct, Type=Dense2026.04 | 25.13 | 44.04 | 46.13 | 15.42 | 18.06 | 83.47 | 38.71 | |
| DASH-KVBackbone=Qwen2-7B-Instruct, Type=Retrieval2026.04 | 24.65 | 44.5 | 45.34 | 15.4 | 19.13 | 83.33 | 38.73 | |
| H2OBackbone=Qwen2.5-14B-Instruct, Type=Eviction2026.04 | 24.4 | 18 | 27.6 | 20.99 | 18.96 | 89.7 | 33.28 | |
| SnapKVBackbone=Qwen2.5-14B-Instruct, Type=Retrieval2026.04 | 24.1 | 20 | 34.6 | 20.9 | 18.97 | 90 | 34.76 | |
| SnapKVBackbone=Qwen2-7B-Instruct, Type=Retrieval2026.04 | 23.86 | 15.6 | 38.61 | 23.07 | 24.56 | 89.31 | 35.84 | |
| H2OBackbone=Llama-3.1-8B-Instruct, Type=Eviction2026.04 | 23.1 | 16 | 21.3 | 23.4 | 22.3 | 90.2 | 32.72 | |
| H2OBackbone=Qwen2-7B-Instruct, Type=Eviction2026.04 | 22.88 | 13.3 | 34.28 | 22.72 | 23.69 | 88.75 | 34.27 | |
| SnapKVBackbone=Llama-3.1-8B-Instruct, Type=Retrieval2026.04 | 21.3 | 16.6 | 30.8 | 26.3 | 22.2 | 90.2 | 34.57 | |
| StreamLLMBackbone=Qwen2-7B-Instruct, Type=Eviction2026.04 | 20.47 | 14.31 | 26.97 | 24.88 | 25.7 | 76.56 | 31.48 | |
| StreamLLMBackbone=Qwen2.5-14B-Instruct, Type=Eviction2026.04 | 18.3 | 16.4 | 23.4 | 16.04 | 19.01 | 74.1 | 27.87 | |
| StreamLLMBackbone=Llama-3.1-8B-Instruct, Type=Eviction2026.04 | 13.6 | 11.7 | 20.1 | 24.7 | 21.5 | 71.1 | 27.12 |