Long-context understanding on LongBench-e
57.33AccuracyVanilla
Evaluation results
| Method | Links | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| VanillaBackbone=Qwen-3-8B, Token Reduction=0%2026.04 | 57.33 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SlimInferBackbone=Qwen-3-8B, Token Reduction=40%2026.04 | 54.36 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DASHBackbone=Qwen-3-8B, Token Reduction=40%2026.04 | 52.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LazyLLMBackbone=Qwen-3-8B, Token Reduction=40%2026.04 | 51.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Llama-3.1-8B, Token Reduction=0%2026.04 | 49.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VanillaBackbone=Qwen-2.5-7B-Instruct-1M, Token Reduction=0%2026.04 | 48.87 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SlimInferBackbone=Qwen-2.5-7B-Instruct-1M, Token Reduction=40%2026.04 | 47.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SlimInferBackbone=Llama-3.1-8B, Token Reduction=40%2026.04 | 45.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LazyLLMBackbone=Qwen-2.5-7B-Instruct-1M, Token Reduction=40%2026.04 | 44.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LazyLLMBackbone=Llama-3.1-8B, Token Reduction=40%2026.04 | 44.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DASHBackbone=Qwen-2.5-7B-Instruct-1M, Token Reduction=40%2026.04 | 44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DASHBackbone=Llama-3.1-8B, Token Reduction=40%2026.04 | 43.12 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| D3Backbone=Qwen2.5-7B-Instruct-1M, Prefill-time token reduction setting=Same setting2026.04 | — | 45.54 | 99.33 | — | 85.01 | 40.34 | — | 64.67 | — | 50.16 | 6.19 | — | — | 40.18 | 45 | 44.49 | 60.95 | 6.01 | 27.14 | 15 | |
| DASHBackbone=Qwen2.5-7B-Instruct-1M, Prefill-time token reduction setting=Same setting2026.04 | — | 54.58 | 98 | — | 84.21 | 51.14 | — | 59 | — | 48.03 | 7.01 | — | — | 40.58 | 46.76 | 49.38 | 61 | 6.47 | 31.82 | 16.67 | |
| DenseModel=Qwen2.5-32B-Instruct, Sparse token budget=5122025.06 | — | 54.04 | 99.83 | 69.27 | 86.26 | 36.03 | 43.6 | 75.67 | 52.28 | 60.69 | 30.14 | 22 | 21.91 | 44.08 | 53.52 | — | — | — | — | — | |
| FastVBackbone=Qwen2.5-7B-Instruct-1M, Prefill-time token reduction setting=Same setting2026.04 | — | 52.71 | 83.67 | — | 84.79 | 52.31 | — | 59.33 | — | 43.48 | 6.96 | — | — | 40.44 | 43.99 | 42.63 | 57.67 | 6.53 | 33.98 | 7.33 | |
| HATAModel=Qwen2.5-32B-Instruct, Sparse token budget=5122025.06 | — | 53.9 | 100 | 68.58 | 87.55 | 36.22 | 42.75 | 75.67 | 52.29 | 60.51 | 30.17 | 22 | 21.79 | 43.7 | 53.47 | — | — | — | — | — | |
| LLMLingua2Backbone=Qwen2.5-7B-Instruct-1M, Prefill-time token reduction setting=Same setting2026.04 | — | 49.64 | 99 | — | 80.87 | 53.05 | — | 61.67 | — | 43.29 | 6.44 | — | — | 40.41 | 44.16 | 42.91 | 51.58 | 6.41 | 32.52 | 6.33 | |
| Qwen2.5-7B-Instruct-1MBackbone=Qwen2.5-7B-Instruct-1M, Prefill-time token reduction setting=None2026.04 | — | 59.86 | 99.33 | — | 85.75 | 54.52 | — | 65 | — | 51.07 | 6.97 | — | — | 44.19 | 48.87 | 51.13 | 62.97 | 6.57 | 32.95 | 15 | |
| SnapKV (pr.)Backbone=Qwen2.5-7B-Instruct-1M, Prefill-time token reduction setting=Same setting2026.04 | — | 51.89 | 97.67 | — | 85.25 | 49.37 | — | 63.67 | — | 48.31 | 7 | — | — | 38.14 | 46.15 | 42.98 | 61.54 | 6.7 | 30.87 | 16.6 |