Long-context Question Answering on LongBench
59.71HotPotQA AccuracyFull Attention
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Full Attention2025.10 | 59.71 | — | — | — | 64.93 | — | 54.36 | — | — | |
| Original PromptBackbone=Qwen2.5-7B-Instruct, Token budget=20002026.04 | 57.84 | — | 28.84 | 43.67 | 52.76 | 43.44 | 47.13 | 30.37 | — | |
| Full CacheModel=Llama-3.1-8B-Instruct, Cache Budget (B)=Full2026.06 | 55.97 | — | 30.22 | 45.37 | 55.8 | 49.2 | 45 | 31.26 | — | |
| vAttention(oracle-top-k)Backbone=Llama-3.1-8B-Instruct, Context length cap=32K, New tokens cap=100, Sparsity=10%2025.10 | 55.95 | — | 29.17 | 47.79 | 54.46 | 41.33 | 46.27 | 30.57 | 25.1 | |
| full attentionBackbone=Llama-3.1-8B-Instruct, Context length cap=32K, New tokens cap=100, Sparsity=10%2025.10 | 55.75 | — | 29.25 | 46.87 | 54.25 | 41.24 | 46.51 | 31.07 | 24.98 | |
| CacheCliprecomp%=20%, auxiliary model=Qwen2.5-0.5B-Instruct2025.10 | 55.74 | — | — | — | 61.37 | — | 46.04 | — | — | |
| vAttention(HashAttention)Backbone=Llama-3.1-8B-Instruct, Context length cap=32K, New tokens cap=100, Sparsity=10%2025.10 | 55.59 | — | 31.34 | 43.05 | 53.21 | 40.04 | 44.33 | 27.58 | 25.15 | |
| HashAttentionBackbone=Llama-3.1-8B-Instruct, Context length cap=32K, New tokens cap=100, Sparsity=10%2025.10 | 52.57 | — | 27.15 | 43.96 | 53.79 | 38.64 | 45.5 | 23.13 | 24.38 | |
| CacheCliprecomp%=20%, auxiliary model=SmolLM2-135M-Instruct2025.10 | 52.12 | — | — | — | 62.43 | — | 41.4 | — | — | |
| oracle-top-kBackbone=Llama-3.1-8B-Instruct, Context length cap=32K, New tokens cap=100, Sparsity=10%2025.10 | 51.98 | — | 30.01 | 44.48 | 52.95 | 39.35 | 47.28 | 23.29 | 25.45 | |
| DensePer-Query Budget=-, Backbone=SDAR-8B-Chat2026.04 | 49.35 | 85.72 | 19.06 | 18.25 | 49.49 | 44.37 | — | — | — | |
| LoSAPer-Query Budget=1024, Backbone=SDAR-8B-Chat2026.04 | 47.84 | 85.93 | 18.44 | 14.7 | 48.43 | 43.07 | — | — | — | |
| LoSAPer-Query Budget=512, Backbone=SDAR-8B-Chat2026.04 | 47.77 | 83.29 | 17.37 | 14.12 | 49.9 | 42.49 | — | — | — | |
| COREBackbone=Qwen2.5-7B-Instruct, Token budget=20002026.04 | 46.28 | — | 17.32 | 36.62 | 45.52 | 35.25 | 39.81 | 25.97 | — | |
| K-VECModel=Llama-3.1-8B-Instruct, Cache Budget (B)=10242026.06 | 45.75 | — | 24.75 | 41.77 | 42.99 | 36.33 | 40.24 | 22.45 | — | |
| LoSAPer-Query Budget=256, Backbone=SDAR-8B-Chat2026.04 | 45.68 | 83.16 | 15.65 | 13.17 | 48.33 | 41.2 | — | — | — | |
| APE2025.10 | 45.29 | — | — | — | 59.7 | — | 38.34 | — | — | |
| Direct Reuse2025.10 | 44.63 | — | — | — | 49.44 | — | 37.7 | — | — | |
| K-VECModel=Llama-3.1-8B-Instruct, Cache Budget (B)=1282026.06 | 44.24 | — | 24.01 | 34.18 | 42.13 | 34.28 | 39.15 | 21.99 | — | |
| CacheBlendrecomp%=20%2025.10 | 44.11 | — | — | — | 57.34 | — | 41.08 | — | — | |
| HeadKV-R2Model=Llama-3.1-8B-Instruct, Cache Budget (B)=1024, Evaluation Protocol=Row 22026.06 | 43.97 | — | 24.66 | 30.82 | 39.56 | 32.95 | 36.47 | 22.24 | — | |
| HeadKV-R2Model=Llama-3.1-8B-Instruct, Cache Budget (B)=1282026.06 | 43.73 | — | 21.8 | 29.19 | 41.89 | 32 | 35.01 | 20.4 | — | |
| HeadKV-R2Model=Llama-3.1-8B-Instruct, Cache Budget (B)=1024, Evaluation Protocol=Row 12026.06 | 43.73 | — | 21.8 | 29.19 | 41.89 | 32 | 35.01 | 20.4 | — | |
| SnapKVModel=Llama-3.1-8B-Instruct, Cache Budget (B)=10242026.06 | 43.4 | — | 25.76 | 27.5 | 38.38 | 31.65 | 34.81 | 20.07 | — | |
| LoSAPer-Query Budget=128, Backbone=SDAR-8B-Chat2026.04 | 43.36 | 80.32 | 18.69 | 15.63 | 46.74 | 40.95 | — | — | — | |
| HeadKV-RModel=Llama-3.1-8B-Instruct, Cache Budget (B)=1282026.06 | 42.45 | — | 23.49 | 25.39 | 38.15 | 30.38 | 32.84 | 19.95 | — | |
| SnapKVModel=Llama-3.1-8B-Instruct, Cache Budget (B)=1282026.06 | 41.12 | — | 22.11 | 15.79 | 31.01 | 26.43 | 29.2 | 19.35 | — | |
| LongLLMLinguaBackbone=Qwen2.5-7B-Instruct, Token budget=20002026.04 | 38.79 | — | 12.27 | 27 | 30.41 | 28.51 | 41.96 | 20.6 | — | |
| EASE-TTTModel=Qwen3-1.7B2026.06 | 36.6 | — | 16.3 | 39.2 | 44.6 | 30.6 | 32.1 | 14.9 | — | |
| COREBackbone=LLaMA2-7B-chat-4k, Token budget=20002026.04 | 35.17 | — | 19.54 | 23.98 | 40.3 | 27.56 | 29.99 | 16.36 | — | |
| LLMLingua-2Backbone=Qwen2.5-7B-Instruct, Token budget=20002026.04 | 35.06 | — | 12.21 | 42.2 | 39.55 | 30 | 37.12 | 13.87 | — | |
| COREBackbone=Longchat1.5-7B-32k, Token budget=20002026.04 | 34.21 | — | 18.41 | 28.99 | 41.73 | 27.93 | 25.61 | 18.61 | — | |
| ICRModel=Qwen3-1.7B2026.06 | 33.6 | — | 13.8 | 27.1 | 43.3 | 27.6 | 31.7 | 16 | — | |
| qTTTModel=Qwen3-1.7B2026.06 | 33.4 | — | 16.2 | 37.2 | 43.3 | 28.7 | 28.2 | 13.6 | — | |
| QUESTPer-Query Budget=1024, Backbone=SDAR-8B-Chat2026.04 | 32.59 | 80.92 | 11 | 19.28 | 42.3 | 37.22 | — | — | — | |
| QUESTPer-Query Budget=256, Backbone=SDAR-8B-Chat2026.04 | 32.57 | 78.36 | 8.73 | 18.21 | 43.2 | 36.21 | — | — | — | |
| QUESTPer-Query Budget=512, Backbone=SDAR-8B-Chat2026.04 | 32.5 | 78.43 | 9.8 | 19.36 | 43.08 | 36.63 | — | — | — | |
| RAGModel=Qwen3-1.7B2026.06 | 31.4 | — | 16.7 | 28.9 | 39.1 | 25.3 | 22.7 | 12.7 | — | |
| Original PromptBackbone=Longchat1.5-7B-32k, Token budget=20002026.04 | 31.1 | — | 20.07 | 28.17 | 41.72 | 24.98 | 19.37 | 9.46 | — | |
| Full-contextModel=Qwen3-1.7B2026.06 | 30.4 | — | 16.1 | 29.9 | 39.1 | 25 | 22.1 | 12.4 | — | |
| EASE-TTTModel=Llama-3.2-1B2026.06 | 29.3 | — | 16 | 24.3 | 45.6 | 25.8 | 26.5 | 13.2 | — | |
| LongLLMLinguaBackbone=LLaMA2-7B-chat-4k, Token budget=20002026.04 | 28.81 | — | 12.95 | 18.54 | 28 | 22.65 | 31.96 | 15.66 | — | |
| LLMLinguaBackbone=Qwen2.5-7B-Instruct, Token budget=20002026.04 | 27.85 | — | 8.66 | 21 | 22.25 | 20.12 | 27.62 | 13.36 | — | |
| QUESTPer-Query Budget=128, Backbone=SDAR-8B-Chat2026.04 | 27.31 | 70.4 | 6.44 | 17.29 | 41.4 | 32.57 | — | — | — | |
| Original PromptBackbone=LLaMA2-7B-chat-4k, Token budget=20002026.04 | 27.28 | — | 18.64 | 21.9 | 36.63 | 24.01 | 31.96 | 7.65 | — | |
| qTTTModel=Llama-3.2-1B2026.06 | 27.1 | — | 17.3 | 24.8 | 40.8 | 25.3 | 26.3 | 15.4 | — | |
| LLMLingua-2Backbone=Longchat1.5-7B-32k, Token budget=20002026.04 | 25.81 | — | 5.34 | 26.66 | 32.29 | 20.69 | 23.67 | 10.36 | — | |
| LLMLingua-2Backbone=LLaMA2-7B-chat-4k, Token budget=20002026.04 | 25.72 | — | 5.85 | 24.17 | 31.9 | 21.22 | 28.52 | 11.18 | — | |
| LongLLMLinguaBackbone=Longchat1.5-7B-32k, Token budget=20002026.04 | 25.44 | — | 10.08 | 16.82 | 25.6 | 19.78 | 26.55 | 14.2 | — | |
| ICRModel=Llama-3.2-1B2026.06 | 25.3 | — | 17.1 | 15.7 | 39.8 | 23.3 | 26.6 | 15.1 | — | |
| LLMLinguaBackbone=LLaMA2-7B-chat-4k, Token budget=20002026.04 | 24.55 | — | 11.43 | 14.86 | 24.32 | 17.01 | 20.28 | 6.59 | — | |
| qTTTModel=Qwen3-0.6B2026.06 | 23.6 | — | 12.1 | 29.8 | 39.5 | 22.4 | 20.2 | 8.9 | — | |
| EASE-TTTModel=Qwen3-0.6B2026.06 | 23.5 | — | 14 | 32.1 | 40.4 | 23.6 | 22.1 | 9.2 | — | |
| RAGModel=Llama-3.2-1B2026.06 | 22 | — | 16.5 | 21 | 33.9 | 21.9 | 28.2 | 9.9 | — | |
| LLMLinguaBackbone=Longchat1.5-7B-32k, Token budget=20002026.04 | 21.89 | — | 10.44 | 16.59 | 21.39 | 16.03 | 18.82 | 7.05 | — | |
| ICRModel=Qwen3-0.6B2026.06 | 21.3 | — | 9.8 | 11.3 | 38.5 | 18.1 | 21.4 | 6.3 | — | |
| Full-contextModel=Llama-3.2-1B2026.06 | 19.1 | — | 14 | 16.2 | 28.9 | 19.7 | 28.9 | 11.1 | — | |
| Full-contextModel=Qwen3-0.6B2026.06 | 17.9 | — | 9.6 | 26 | 38.8 | 19.5 | 17 | 8 | — | |
| RAGModel=Qwen3-0.6B2026.06 | 17.9 | — | 11.6 | 26.2 | 37.6 | 19.6 | 17.2 | 7.2 | — |