Long-context understanding on LongBench (test)
58.7Avg ScoreLlama3.1-8B
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Llama3.1-8BCache Bits (K,V,Index)=16, 16, 0, Backbone=Llama3.1-8B2026.03 | 58.7 | — | 45.5 | — | 54.7 | 47.1 | — | 34.9 | 25.3 | — | — | — | — | — | — | — | — | 53.8 | 73 | 91.6 | — | — | 99.5 | 63.4 | — | — | — | — | — | 56.7 | — | — | — | — | — | — | — | |
| Self-Indexing KVCacheCache Bits (K,V,Index)=16, 16, 1, Backbone=Llama3.1-8B2026.03 | 58.4 | — | 45.4 | — | 55.4 | 47 | — | 34 | 24.8 | — | — | — | — | — | — | — | — | 53.9 | 72.5 | 91.7 | — | — | 99.5 | 63.2 | — | — | — | — | — | 55.3 | — | — | — | — | — | — | — | |
| Self-Indexing KVCacheCache Bits (K,V,Index)=2, 2, 1, Backbone=Llama3.1-8B2026.03 | 58.2 | — | 44.8 | — | 55.7 | 46 | — | 33.1 | 24.7 | — | — | — | — | — | — | — | — | 54.6 | 72.5 | 92.1 | — | — | 99.5 | 63.2 | — | — | — | — | — | 54.4 | — | — | — | — | — | — | — | |
| Qwen2.5-14BCache Bits (K,V,Index)=16, 16, 0, Backbone=Qwen2.5-14B2026.03 | 56.9 | — | 46.5 | — | 65 | 64.3 | — | 31.9 | 23.6 | — | — | — | — | — | — | — | — | 54 | 81 | 89 | — | — | 100 | 33.1 | — | — | — | — | — | 38.1 | — | — | — | — | — | — | — | |
| QuestCache Bits (K,V,Index)=16, 16, 2, Backbone=Llama3.1-8B2026.03 | 56.5 | — | 44.9 | — | 55 | 46.9 | — | 31.2 | 24 | — | — | — | — | — | — | — | — | 51.6 | 71 | 90.6 | — | — | 95.5 | 60.6 | — | — | — | — | — | 50.5 | — | — | — | — | — | — | — | |
| Self-Indexing KVCacheCache Bits (K,V,Index)=16, 16, 1, Backbone=Qwen2.5-14B2026.03 | 55.9 | — | 46 | — | 63.7 | 62.4 | — | 30.5 | 22.6 | — | — | — | — | — | — | — | — | 53.5 | 80.5 | 88.5 | — | — | 97.5 | 32.6 | — | — | — | — | — | 37.4 | — | — | — | — | — | — | — | |
| Self-Indexing KVCacheCache Bits (K,V,Index)=2, 2, 1, Backbone=Qwen2.5-14B2026.03 | 55.7 | — | 45.8 | — | 63 | 63.1 | — | 30.5 | 22 | — | — | — | — | — | — | — | — | 53.5 | 80.5 | 88.5 | — | — | 96.4 | 32.3 | — | — | — | — | — | 37.4 | — | — | — | — | — | — | — | |
| DoubleSparseCache Bits (K,V,Index)=16, 16, 2, Backbone=Llama3.1-8B2026.03 | 54.8 | — | 43.8 | — | 54.4 | 45.9 | — | 31.7 | 24 | — | — | — | — | — | — | — | — | 51.7 | 67.5 | 91.9 | — | — | 97 | 50.5 | — | — | — | — | — | 44.7 | — | — | — | — | — | — | — | |
| DoubleSparseCache Bits (K,V,Index)=16, 16, 2, Backbone=Qwen2.5-14B2026.03 | 54.1 | — | 44.7 | — | 61 | 62.1 | — | 30.3 | 21.9 | — | — | — | — | — | — | — | — | 53.1 | 79 | 89 | — | — | 94.1 | 27.5 | — | — | — | — | — | 31.8 | — | — | — | — | — | — | — | |
| BF16Model=Llama-3.1-8B-Instruct, Bit-width=BF162025.12 | 54 | — | 45.53 | — | — | — | — | — | 25.36 | 27.19 | — | — | — | — | — | — | — | 56.37 | 72.5 | 91.65 | 43.62 | — | — | 65.1 | — | — | — | — | — | 58.65 | — | — | — | — | — | — | — | |
| QuestCache Bits (K,V,Index)=16, 16, 2, Backbone=Qwen2.5-14B2026.03 | 53.9 | — | 45.7 | — | 62.6 | 62.3 | — | 28.8 | 21.3 | — | — | — | — | — | — | — | — | 52 | 70.5 | 88.1 | — | — | 94.5 | 31.7 | — | — | — | — | — | 35.2 | — | — | — | — | — | — | — | |
| BF16Model=Mistral-7B-Instruct-v0.3, Bit-width=BF162025.12 | 53.84 | — | 41.58 | — | — | — | — | — | 25.75 | 27.82 | — | — | — | — | — | — | — | 55.23 | 76 | 88.59 | 47.35 | — | — | 60.53 | — | — | — | — | — | 61.68 | — | — | — | — | — | — | — | |
| Llama-3.1-8BTokens=128k, Backbone=Llama-3.1-8B2025.05 | 53.8 | — | 45.5 | — | 54.7 | 47.1 | — | 34.9 | — | 27.5 | — | — | — | — | — | — | — | 53.8 | 73 | 91.6 | 43.8 | 7.5 | 56.7 | 63.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MixKVQModel=Llama-3.1-8B-Instruct, Bit-width=C2.72025.12 | 53.71 | — | 45.32 | — | — | — | — | — | 25.29 | 27.2 | — | — | — | — | — | — | — | 55.4 | 72.5 | 91.78 | 43.68 | — | — | 64.96 | — | — | — | — | — | 57.26 | — | — | — | — | — | — | — | |
| MixKVQModel=Mistral-7B-Instruct-v0.3, Bit-width=C2.72025.12 | 53.68 | — | 41.19 | — | — | — | — | — | 26.23 | 27.66 | — | — | — | — | — | — | — | 54.52 | 76 | 88.59 | 46.89 | — | — | 60.44 | — | — | — | — | — | 61.6 | — | — | — | — | — | — | — | |
| TailorKV-2Tokens=64(+128), Backbone=Llama-3.1-8B2025.05 | 52.9 | — | 44.8 | — | 54.8 | 46.2 | — | 31.9 | — | 26.8 | — | — | — | — | — | — | — | 53.9 | 70.5 | 92.3 | 43.2 | 7.7 | 54.2 | 62.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| RotateKVModel=Llama-3.1-8B-Instruct, Bit-width=KV42025.12 | 52.89 | — | 44.32 | — | — | — | — | — | 25 | 27.03 | — | — | — | — | — | — | — | 53.92 | 71.5 | 90.84 | 43.12 | — | — | 63.88 | — | — | — | — | — | 56.39 | — | — | — | — | — | — | — | |
| KIVIModel=Mistral-7B-Instruct-v0.3, Bit-width=KV42025.12 | 52.75 | — | 40.87 | — | — | — | — | — | 23.9 | 26.94 | — | — | — | — | — | — | — | 53.68 | 74 | 88.34 | 47.54 | — | — | 58.64 | — | — | — | — | — | 60.81 | — | — | — | — | — | — | — | |
| KIVIModel=Llama-3.1-8B-Instruct, Bit-width=KV42025.12 | 52.68 | — | 44.42 | — | — | — | — | — | 25.28 | 26.86 | — | — | — | — | — | — | — | 52.1 | 72.5 | 91.42 | 44.07 | — | — | 62.45 | — | — | — | — | — | 55.04 | — | — | — | — | — | — | — | |
| TailorKV-1Tokens=64(+128), Backbone=Llama-3.1-8B2025.05 | 52.6 | — | 43.4 | — | 55.3 | 46.5 | — | 31.3 | — | 27.2 | — | — | — | — | — | — | — | 53 | 70 | 91.6 | 42.5 | 7.9 | 54.8 | 61.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SKVQModel=Llama-3.1-8B-Instruct, Bit-width=KV42025.12 | 52.54 | — | 44.21 | — | — | — | — | — | 25.34 | 26.79 | — | — | — | — | — | — | — | 53.96 | 72.5 | 91.14 | 42.38 | — | — | 62.44 | — | — | — | — | — | 54.13 | — | — | — | — | — | — | — | |
| KVQuantModel=Mistral-7B-Instruct-v0.3, Bit-width=KV42025.12 | 52.43 | — | 40.56 | — | — | — | — | — | 24.72 | 26.79 | — | — | — | — | — | — | — | 53.41 | 75 | 88.21 | 46.97 | — | — | 58.08 | — | — | — | — | — | 58.13 | — | — | — | — | — | — | — | |
| SKVQModel=Mistral-7B-Instruct-v0.3, Bit-width=KV42025.12 | 52.37 | — | 40.95 | — | — | — | — | — | 22.04 | 26.57 | — | — | — | — | — | — | — | 53.36 | 76 | 89.32 | 47.65 | — | — | 58.85 | — | — | — | — | — | 56.58 | — | — | — | — | — | — | — | |
| KVQuantModel=Llama-3.1-8B-Instruct, Bit-width=KV42025.12 | 52.3 | — | 44.03 | — | — | — | — | — | 25.25 | 26.87 | — | — | — | — | — | — | — | 51.67 | 71 | 91.03 | 43.61 | — | — | 62.15 | — | — | — | — | — | 55.13 | — | — | — | — | — | — | — | |
| ClassicSD / MagicDecTarget Model=Qwen2.5-14B-Instruct, KV Budget=None2026.06 | 52.27 | — | — | — | — | — | — | — | — | — | 42.16 | 52.95 | 27.36 | 71.64 | 55.88 | 59.29 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PQCacheTokens=192, Backbone=Llama-3.1-8B2025.05 | 51.7 | — | 45.6 | — | 53.8 | 45.3 | — | 29 | — | 25.9 | — | — | — | — | — | — | — | 51.2 | 69.5 | 91.3 | 41.2 | 8.2 | 54.1 | 58.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DustinTarget Model=Qwen2.5-14B-Instruct, KV Budget=5122026.06 | 51.46 | — | — | — | — | — | — | — | — | — | 41.11 | 52.36 | 26.68 | 70.88 | 55.7 | 58.15 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ClassicSD / MagicDecTarget Model=Qwen2.5-7B-Instruct, KV Budget=None2026.06 | 51.46 | — | — | — | — | — | — | — | — | — | 40.54 | 44.28 | 27.76 | 69.92 | 53.5 | 64.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KIVIModel=Mistral-7B-Instruct-v0.3, Bit-width=KV22025.12 | 51.1 | — | 38.56 | — | — | — | — | — | 22.48 | 26.65 | — | — | — | — | — | — | — | 50.47 | 72 | 87.13 | 47.18 | — | — | 57.15 | — | — | — | — | — | 58.29 | — | — | — | — | — | — | — | |
| KIVIModel=Llama-3.1-8B-Instruct, Bit-width=KV22025.12 | 50.72 | — | 41.02 | — | — | — | — | — | 24.57 | 26.79 | — | — | — | — | — | — | — | 45.86 | 71.5 | 91.14 | 43.41 | — | — | 60.55 | — | — | — | — | — | 51.64 | — | — | — | — | — | — | — | |
| DustinTarget Model=Qwen2.5-7B-Instruct, KV Budget=5122026.06 | 50.4 | — | — | — | — | — | — | — | — | — | 40.21 | 43.83 | 27.45 | 68.75 | 53.5 | 61.45 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DustinTarget Model=Qwen2.5-14B-Instruct, KV Budget=2562026.06 | 50.31 | — | — | — | — | — | — | — | — | — | 40.21 | 51.19 | 26.18 | 69.88 | 54.42 | 56.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SKVQModel=Mistral-7B-Instruct-v0.3, Bit-width=KV22025.12 | 49.95 | — | 32.64 | — | — | — | — | — | 22.2 | 26.46 | — | — | — | — | — | — | — | 46.57 | 76 | 87.52 | 46.09 | — | — | 57.9 | — | — | — | — | — | 54.16 | — | — | — | — | — | — | — | |
| SKVQModel=Llama-3.1-8B-Instruct, Bit-width=KV22025.12 | 49.95 | — | 35.28 | — | — | — | — | — | 23.61 | 26.47 | — | — | — | — | — | — | — | 46.27 | 72.5 | 91.21 | 42.96 | — | — | 60.63 | — | — | — | — | — | 50.63 | — | — | — | — | — | — | — | |
| KVQuantModel=Mistral-7B-Instruct-v0.3, Bit-width=KV22025.12 | 49.94 | — | 37.88 | — | — | — | — | — | 22.16 | 25.89 | — | — | — | — | — | — | — | 49.27 | 70 | 86.53 | 45.38 | — | — | 56.42 | — | — | — | — | — | 55.94 | — | — | — | — | — | — | — | |
| SnapKVCache Bits (K,V,Index)=16, 16, 0, Backbone=Llama3.1-8B2026.03 | 49.9 | — | 33.9 | — | 54.4 | 44.7 | — | 21.8 | 22.6 | — | — | — | — | — | — | — | — | 46.4 | 48 | 90.5 | — | — | 78 | 58 | — | — | — | — | — | 50.1 | — | — | — | — | — | — | — | |
| RotateKVModel=Mistral-7B-Instruct-v0.3, Bit-width=KV42025.12 | 49.73 | — | 40.64 | — | — | — | — | — | 22.9 | 25.09 | — | — | — | — | — | — | — | 53.59 | 76 | 88.16 | 44.08 | — | — | 49.22 | — | — | — | — | — | 47.92 | — | — | — | — | — | — | — | |
| FullKVBackbone=Qwen3-8B, KV Cache Size=Full2026.03 | 49.46 | — | 47.76 | — | 59.23 | 43.37 | 36.05 | 33.66 | 24.05 | 24.79 | — | — | — | — | — | — | — | 53.33 | 71.5 | 90.21 | 44.43 | — | — | 69.39 | — | — | — | — | — | 65.57 | — | — | — | — | 26.04 | 2 | 100 | |
| DustinTarget Model=Qwen2.5-7B-Instruct, KV Budget=2562026.06 | 49.44 | — | — | — | — | — | — | — | — | — | 40.41 | 43.17 | 27.38 | 67.89 | 49.25 | 60.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=20482026.03 | 49.33 | — | 48.01 | — | 59.43 | 43.2 | 36.21 | 32.64 | 23.93 | 24.93 | — | — | — | — | — | — | — | 52.92 | 71 | 90.21 | 44.74 | — | — | 69.23 | — | — | — | — | — | 65.01 | — | — | — | — | 26.76 | 1 | 100 | |
| SpecKVBackbone=Qwen3-8B, KV Cache Size=20482026.03 | 49.17 | — | 47 | — | 61.43 | 43.74 | 34.94 | 31.38 | 23.98 | 24.57 | — | — | — | — | — | — | — | 53.8 | 70.5 | 91.11 | 44.46 | — | — | 69.12 | — | — | — | — | — | 65.87 | — | — | — | — | 24.86 | 0 | 100 | |
| LAQBackbone=Qwen3-8B, KV Cache Size=20482026.03 | 49.15 | — | 47.22 | — | 57.72 | 43.45 | 37.4 | 31.24 | 24.03 | 24.51 | — | — | — | — | — | — | — | 53.71 | 72.5 | 90.13 | 44.54 | — | — | 68.29 | — | — | — | — | — | 64.68 | — | — | — | — | 24.94 | 2 | 100 | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=10242026.03 | 49.04 | — | 47.23 | — | 59.3 | 43.25 | 36.39 | — | 23.72 | 24.61 | — | — | — | — | — | — | — | 52.56 | 71 | 90.21 | 44.69 | — | — | 68.93 | — | — | — | — | — | 65.22 | — | — | — | — | 25.36 | 0.5 | 100 | |
| SpecKVBackbone=Qwen3-8B, KV Cache Size=10242026.03 | 48.98 | — | 46.56 | — | 59.04 | 43.37 | 34.12 | 29.32 | 24.18 | 23.68 | — | — | — | — | — | — | — | 54.07 | 71 | 90.11 | 44.56 | — | — | 69.09 | — | — | — | — | — | 66.53 | — | — | — | — | 24.98 | 3 | 100 | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=20482026.03 | 48.96 | — | 47.52 | — | 58.73 | 42.7 | 36.08 | 30.64 | 23.78 | 24.4 | — | — | — | — | — | — | — | 53.2 | 71.5 | 90.21 | 43.27 | — | — | 69.33 | — | — | — | — | — | 65.36 | — | — | — | — | 25.55 | 1.1 | 100 | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=5122026.03 | 48.93 | — | 46.49 | — | 59.32 | 43.09 | 36.92 | 29.56 | 23.8 | 24.01 | — | — | — | — | — | — | — | 52.04 | 71.5 | 90.21 | 44.2 | — | — | 68.88 | — | — | — | — | — | 65.58 | — | — | — | — | 25.33 | 2 | 100 | |
| LAQBackbone=Qwen3-8B, KV Cache Size=10242026.03 | 48.84 | — | 47.27 | — | 57.01 | 43.52 | 37.26 | 29.5 | 23.88 | 23.47 | — | — | — | — | — | — | — | 53.45 | 71.5 | 89.63 | 44 | — | — | 67.94 | — | — | — | — | — | 64.83 | — | — | — | — | 26.11 | 2 | 100 | |
| SpecKVBackbone=Qwen3-8B, KV Cache Size=5122026.03 | 48.79 | — | 45.65 | — | 60.53 | 43.85 | 35.26 | 27.21 | 24.04 | 22.53 | — | — | — | — | — | — | — | 54.34 | 70.5 | 90.2 | 43.71 | — | — | 69.25 | — | — | — | — | — | 65.85 | — | — | — | — | 24.22 | 3.5 | 100 | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=20482026.03 | 48.78 | — | 46.69 | — | 58.41 | 42.9 | 36.61 | 29.41 | 23.61 | 24.18 | — | — | — | — | — | — | — | 53.21 | 71.5 | 90.05 | 42.9 | — | — | 69.28 | — | — | — | — | — | 65.21 | — | — | — | — | 25.47 | 1.1 | 100 | |
| LAQBackbone=Qwen3-8B, KV Cache Size=5122026.03 | 48.77 | — | 46.98 | — | 57.31 | 43.35 | 37.64 | 26.93 | 23.67 | 22.19 | — | — | — | — | — | — | — | 53.7 | 72.5 | 88.96 | 43.81 | — | — | 68.23 | — | — | — | — | — | 64.71 | — | — | — | — | 27.34 | 3 | 100 | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=10242026.03 | 48.45 | — | 46.13 | — | 58.52 | 42.66 | 36.89 | 28.39 | 23.61 | 23.33 | — | — | — | — | — | — | — | 52.48 | 69 | 89.55 | 43.13 | — | — | 69.05 | — | — | — | — | — | 66.27 | — | — | — | — | 24.26 | 2 | 100 | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 48.31 | — | 45.4 | — | 58.47 | 44.05 | 36.13 | 27.77 | 23.71 | 22.88 | — | — | — | — | — | — | — | 52.68 | 69 | 89.05 | 43.32 | — | — | 67.83 | — | — | — | — | — | 64.71 | — | — | — | — | 25.88 | 2 | 100 | |
| TriAttentionModel=Qwen3-8B, KV budget=50%2026.04 | 48.1 | 28.1 | 43 | 51.4 | 60.2 | 44.9 | 36.9 | 32.9 | 23.8 | 24.3 | — | — | — | — | — | — | — | — | 69 | 90.3 | 39.9 | — | 91 | 65 | — | — | — | — | — | 61.3 | — | — | — | — | — | 7 | — | |
| Sketch&WalkBackbone=Llama3.1-8B-Instruct2026.02 | 48 | 30.69 | 44.44 | — | 57.49 | 47.11 | 33.16 | 32.39 | 25.47 | 25.55 | — | — | — | — | — | — | — | 55.73 | 70.5 | 91.4 | 43.39 | 8.75 | 99 | 54.94 | — | — | — | — | — | 47.78 | 50.6 | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=10242026.03 | 48 | — | 42.89 | — | 58.86 | 42.32 | 35.47 | 27.32 | 23.07 | 22.72 | — | — | — | — | — | — | — | 53.01 | 71 | 89.95 | 42.56 | — | — | 68.81 | — | — | — | — | — | 64.25 | — | — | — | — | 23.77 | 2 | 100 | |
| DenseBackbone=Llama3.1-8B-Instruct2026.02 | 47.95 | 30.05 | 44.67 | — | 55.4 | 45.62 | 29.41 | 34.77 | 25.14 | 26.9 | — | — | — | — | — | — | — | 55.97 | 73 | 91.16 | 43.24 | 10 | 99 | 55.13 | — | — | — | — | — | 47.79 | 50.48 | — | — | — | — | — | — | |
| LAQBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 47.95 | — | 45.44 | — | 57 | 43.53 | 36.62 | 24.22 | 23.38 | 20.38 | — | — | — | — | — | — | — | 53.84 | 70 | 89.05 | 42.47 | — | — | 68.17 | — | — | — | — | — | 64.03 | — | — | — | — | 26 | 3 | 100 | |
| KVQuantModel=Llama-3.1-8B-Instruct, Bit-width=KV22025.12 | 47.9 | — | 36.51 | — | — | — | — | — | 22.94 | 25.82 | — | — | — | — | — | — | — | 41.85 | 65 | 86.27 | 41.94 | — | — | 59.88 | — | — | — | — | — | 50.87 | — | — | — | — | — | — | — | |
| AdamasBackbone=Llama3.1-8B-Instruct2026.02 | 47.84 | 30.09 | 43.84 | — | 56.07 | 45.81 | 31.2 | 34.72 | 25.05 | 26.89 | — | — | — | — | — | — | — | 56.2 | 73 | 91.29 | 43.14 | 8 | 99 | 54.2 | — | — | — | — | — | 46.96 | 50.49 | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=5122026.03 | 47.64 | — | 43.72 | — | 58.37 | 42.36 | 34.04 | 25.03 | 22.55 | 21.66 | — | — | — | — | — | — | — | 51.96 | 69 | 89.53 | 42.06 | — | — | 69.37 | — | — | — | — | — | 64.96 | — | — | — | — | 24.63 | 3 | 100 | |
| DustinTarget Model=Qwen2.5-14B-Instruct, KV Budget=1282026.06 | 47.59 | — | — | — | — | — | — | — | — | — | 40.59 | 49.75 | 25.57 | 69.49 | 38.86 | 53.72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ClassicSD / MagicDecTarget Model=Qwen2.5-32B-Instruct, KV Budget=None2026.06 | 47.52 | — | — | — | — | — | — | — | — | — | 42.43 | 54.53 | 27.37 | 67.57 | 56 | 42.78 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 47.46 | — | 44.3 | — | 58.78 | 42.79 | 35.89 | 25.29 | 22.95 | 21.13 | — | — | — | — | — | — | — | 53.24 | 66.5 | 88.95 | 41.64 | — | — | 65.95 | — | — | — | — | — | 62.88 | — | — | — | — | 26.06 | 3.5 | 99.5 | |
| SpecKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 47.42 | — | 41.09 | — | 59.85 | 42.42 | 34.5 | 24.53 | 23.64 | 21.25 | — | — | — | — | — | — | — | 53.89 | 68 | 88.13 | 43.12 | — | — | 68.39 | — | — | — | — | — | 64.4 | — | — | — | — | 22.58 | 3 | 100 | |
| Full Attn.Model=Qwen3-8B, KV budget=Full2026.04 | 47.2 | 28.8 | 43.8 | 55.3 | 62.8 | 48.9 | 35.5 | 33.5 | 24.7 | 24.7 | — | — | — | — | — | — | — | — | 40.5 | 90.5 | 40.3 | — | 91.8 | 64.9 | — | — | — | — | — | 60 | — | — | — | — | — | 9 | — | |
| Yi-9BTokens=200k, Backbone=Yi-9B2025.05 | 47 | — | 38.4 | — | 52.7 | 36.7 | — | 31 | — | 26.7 | — | — | — | — | — | — | — | 34.9 | 77 | 90 | 14.9 | 2.5 | 67.4 | 71.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QuestBackbone=Llama3.1-8B-Instruct2026.02 | 46.92 | 28.79 | 42.21 | — | 55.93 | 46.95 | 30.89 | 34.8 | 24.19 | 27.32 | — | — | — | — | — | — | — | 55.98 | 69 | 89.9 | 42.64 | 8.03 | 99 | 49.96 | — | — | — | — | — | 45.12 | 49.51 | — | — | — | — | — | — | |
| NSA2025.02 | 46.9 | — | 43.2 | — | 43.7 | 35.6 | — | 30.7 | — | — | — | — | — | — | — | — | — | 50.3 | — | — | — | — | 90.5 | 23.2 | 62.4 | 0.341 | 55 | — | — | — | — | — | — | — | — | — | — | |
| DustinTarget Model=Qwen2.5-32B-Instruct, KV Budget=5122026.06 | 46.85 | — | — | — | — | — | — | — | — | — | 41.37 | 54.5 | 26.6 | 66.75 | 56.25 | 41.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LAQBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 46.52 | — | 42.15 | — | 57.89 | 42.84 | 36.74 | 21.33 | 22.25 | 18.34 | — | — | — | — | — | — | — | 53.55 | 64.5 | 89.55 | 40.93 | — | — | 66.74 | — | — | — | — | — | 61.7 | — | — | — | — | 22.74 | 3 | 100 | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=5122026.03 | 46.44 | — | 40.52 | — | 57.57 | 40.89 | 32.85 | 23.85 | 21.92 | 19.7 | — | — | — | — | — | — | — | 51.43 | 68.5 | 89.55 | 40.89 | — | — | 67.47 | — | — | — | — | — | 61.73 | — | — | — | — | 23.12 | 3 | 100 | |
| DustinTarget Model=Qwen2.5-7B-Instruct, KV Budget=1282026.06 | 46.29 | — | — | — | — | — | — | — | — | — | 39.87 | 42.75 | 26.76 | 67.84 | 35.5 | 55.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QuestTokens=192, Backbone=Llama-3.1-8B2025.05 | 46.2 | — | 35.9 | — | 52.8 | 41 | — | 17.7 | — | 23.8 | — | — | — | — | — | — | — | 44.2 | 63 | 86 | 36 | 8.4 | 43.6 | 52.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DustinTarget Model=Qwen2.5-32B-Instruct, KV Budget=2562026.06 | 45.72 | — | — | — | — | — | — | — | — | — | 41.29 | 54.1 | 25.71 | 64.87 | 53.5 | 40.53 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SpecKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 45.62 | — | 37.14 | — | 56.77 | 42.24 | 31.82 | 21.33 | 22.86 | 19.04 | — | — | — | — | — | — | — | 53.58 | 60 | 88.31 | 41.5 | — | — | 66.82 | — | — | — | — | — | 61.96 | — | — | — | — | 23.03 | 3.5 | 100 | |
| Ada-KV+SnapKVModel=Qwen3-8B, KV budget=50%2026.04 | 45.6 | 27 | 36.8 | 45 | 59.6 | 47.5 | 34.1 | 31.8 | 23 | 23.6 | — | — | — | — | — | — | — | — | 46 | 90.1 | 40.9 | — | 90.8 | 65.6 | — | — | — | — | — | 59.6 | — | — | — | — | — | 8 | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 45.45 | — | 38.32 | — | 57.36 | 40.67 | 32.82 | 21.51 | 21.89 | 18.97 | — | — | — | — | — | — | — | 51.04 | 59.5 | 89.46 | 41.06 | — | — | 67.62 | — | — | — | — | — | 61.88 | — | — | — | — | 23.03 | 2 | 100 | |
| SnapKVTokens=192, Backbone=Llama-3.1-8B2025.05 | 45.2 | — | 25.7 | — | 52.6 | 43.7 | — | 20 | — | 20.5 | — | — | — | — | — | — | — | 44.7 | 41 | 89 | 39.6 | 8 | 48.7 | 57 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVModel=Qwen3-8B, KV budget=50%2026.04 | 45.2 | 26.9 | 37 | 45.5 | 59.2 | 46.3 | 33.1 | 32.2 | 23 | 23.4 | — | — | — | — | — | — | — | — | 40.5 | 89.9 | 41 | — | 91.1 | 66.4 | — | — | — | — | — | 59.9 | — | — | — | — | — | 7.5 | — | |
| GPT-4-Turboparadigm=Zero-shot, truncation=Middle truncation strategy, decoding=Greedy decoding, context_limit=32K2024.04 | 44.7 | 33 | 50.7 | 52.7 | 68.5 | 64.3 | 49.1 | 33.9 | 25.4 | 24.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TailorKV-1Tokens=64(+128), Backbone=Yi-9B2025.05 | 44.7 | — | 37.7 | — | 52.8 | 35.8 | — | 29.8 | — | 24.9 | — | — | — | — | — | — | — | 38.2 | 76 | 89.5 | 15.2 | 3.5 | 64.3 | 68.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SnapKVCache Bits (K,V,Index)=16, 16, 0, Backbone=Qwen2.5-14B2026.03 | 44.3 | — | 29.4 | — | 60.1 | 54 | — | 19.3 | 19.8 | — | — | — | — | — | — | — | — | 47.6 | 54.5 | 86 | — | — | 61 | 24.9 | — | — | — | — | — | 31.1 | — | — | — | — | — | — | — | |
| TailorKV-2Tokens=64(+128), Backbone=Yi-9B2025.05 | 44 | — | 37.6 | — | 52.6 | 34.4 | — | 29.4 | — | 25.3 | — | — | — | — | — | — | — | 33.6 | 76 | 89.3 | 15 | 3 | 63.5 | 68.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Full Attention2025.02 | 43.7 | — | 40.9 | — | 35 | 30.5 | — | 32.4 | — | — | — | — | — | — | — | — | — | 51.2 | — | — | — | — | 83 | 16.3 | 62.3 | 0.294 | 56 | — | — | — | — | — | — | — | — | — | — | |
| LOOKAHEADKVBackbone=Qwen3-8B, KV Cache Size=642026.03 | 43.53 | — | 43.13 | — | 59.01 | 42.5 | 34.34 | 22.66 | 21.61 | 18.49 | — | — | — | — | — | — | — | 51.85 | 64.5 | 88.75 | 39.48 | — | — | 62.78 | — | — | — | — | — | 56.07 | — | — | — | — | 22.11 | 1.5 | 67.75 | |
| DustinTarget Model=Qwen2.5-32B-Instruct, KV Budget=1282026.06 | 43.28 | — | — | — | — | — | — | — | — | — | 41.12 | 53.7 | 24.76 | 62.81 | 37.92 | 39.77 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen3-8B, KV Cache Size=2562026.03 | 43.05 | — | 34.87 | — | 55.68 | 37.89 | 26.67 | 20.43 | 20.92 | 17.43 | — | — | — | — | — | — | — | 47.44 | 58.5 | 85.2 | 38.98 | — | — | 65.51 | — | — | — | — | — | 57.32 | — | — | — | — | 18.47 | 3.5 | 100 | |
| PyramidKVModel=Qwen3-8B, KV budget=50%2026.04 | 42.7 | 25.9 | 30.4 | 39.1 | 52.2 | 39.9 | 29.7 | 29.9 | 22.2 | 21.9 | — | — | — | — | — | — | — | — | 33.5 | 90 | 40.7 | — | 93.4 | 65.7 | — | — | — | — | — | 60.2 | — | — | — | — | — | 8 | — | |
| Exact-Top2025.02 | 42.3 | — | 39.7 | — | 32.1 | 28.8 | — | 31.6 | — | — | — | — | — | — | — | — | — | 50.2 | — | — | — | — | 81 | 15.6 | 60.5 | 0.291 | 54.8 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen3-8B, KV Cache Size=1282026.03 | 42.13 | — | 32.65 | — | 54.81 | 38.95 | 26.59 | 17.66 | 20.83 | 16.04 | — | — | — | — | — | — | — | 45.99 | 49.5 | 87.1 | 38.9 | — | — | 64.62 | — | — | — | — | — | 58.29 | — | — | — | — | 19.14 | 3.5 | 99.5 | |
| PQCacheTokens=192, Backbone=Yi-9B2025.05 | 42 | — | 36.9 | — | 47.6 | 35.5 | — | 19.3 | — | 19.2 | — | — | — | — | — | — | — | 27.9 | 74 | 89.6 | 12.2 | 4.6 | 62 | 66.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| StreamingLLMBackbone=Qwen3-8B, KV Cache Size=20482026.03 | 41.47 | — | 38.54 | — | 47.96 | 36.78 | 25.53 | 28.69 | 21.47 | 24.11 | — | — | — | — | — | — | — | 38.02 | 65 | 90.3 | 42.85 | — | — | 68.37 | — | — | — | — | — | 64.63 | — | — | — | — | 21.73 | 1 | 48.5 | |
| StreamLLMTokens=192, Backbone=Llama-3.1-8B2025.05 | 40.6 | — | 21.4 | — | 46.5 | 38.9 | — | 17.9 | — | 18 | — | — | — | — | — | — | — | 31.3 | 40 | 75.7 | 34.4 | 8 | 45.7 | 50.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QuestTarget Model=Qwen2.5-32B-Instruct, KV Budget=5122026.06 | 40.41 | — | — | — | — | — | — | — | — | — | 31.3 | 36.88 | 24.16 | 59.55 | 54.97 | 40 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FILM-7Bparadigm=Zero-shot, truncation=Middle truncation strategy, decoding=Greedy decoding, context_limit=32K2024.04 | 39.9 | 26.9 | 42.2 | 56 | 62.1 | 47 | 39 | 33.8 | 25.1 | 26.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QuestTarget Model=Qwen2.5-14B-Instruct, KV Budget=5122026.06 | 39.83 | — | — | — | — | — | — | — | — | — | 29.27 | 30.51 | 25.04 | 60.66 | 49.73 | 43.67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| StreamingLLMModel=Qwen3-8B, KV budget=50%2026.04 | 39.4 | 24.1 | 30.5 | 31.2 | 46.5 | 41.6 | 21.6 | 30.8 | 21.8 | 23.9 | — | — | — | — | — | — | — | — | 43 | 85.4 | 38.2 | — | 55.2 | 64.7 | — | — | — | — | — | 61.2 | — | — | — | — | — | 10 | — | |
| Quest2025.02 | 39.2 | — | 36.5 | — | 29.5 | 24.5 | — | 29.3 | — | — | — | — | — | — | — | — | — | 49.5 | — | — | — | — | 79.2 | 13.5 | 56.1 | 0.257 | 47.8 | — | — | — | — | — | — | — | — | — | — | |
| StreamingLLMBackbone=Qwen3-8B, KV Cache Size=10242026.03 | 39.19 | — | 32.82 | — | 45.94 | 34.38 | 23.34 | 25.73 | 20.25 | 23.5 | — | — | — | — | — | — | — | 31.44 | 62 | 88.71 | 41.18 | — | — | 68.39 | — | — | — | — | — | 63.65 | — | — | — | — | 21.25 | 0.5 | 44 | |
| QuestTokens=192, Backbone=Yi-9B2025.05 | 39.1 | — | 32.4 | — | 44.2 | 31.5 | — | 14.3 | — | 16.5 | — | — | — | — | — | — | — | 26 | 73 | 86.2 | 13.4 | 3.9 | 55.6 | 63.7 | — | — | — | — | — | — | — | — | — | — | — | — | — |