Long-context understanding on LongBench 1.0 (test)
32.94NarrativeQALaProx
Evaluation results
| Method | Links | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LaProxBackbone=Qwen3-30B-A3B-Instruct-2507, KV Cache Budget=20% Cache2026.05 | 32.94 | 37.41 | 46.1 | 59.49 | 47.43 | 25.38 | 28.82 | 19.97 | 22.07 | 78 | 87.29 | 48.54 | 75.78 | 71.74 | 12 | 100 | 49.56 | — | — | — | — | — | — | — | — | — | — | |
| CriticalKVBackbone=Qwen3-30B-A3B-Instruct-2507, KV Cache Budget=20% Cache2026.05 | 32.06 | 30.79 | 33.19 | 54.06 | 47.49 | 26.13 | 29.08 | 19.88 | 21.04 | 70 | 88.24 | 47.02 | 75.34 | 69.79 | 12 | 98.5 | 47.16 | — | — | — | — | — | — | — | — | — | — | |
| Nexus SamplingModel=Llama-3.1-8B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 30.89 | — | 56.78 | 57.61 | 48.06 | 32.37 | 29.44 | 25.23 | 22.15 | 70 | 92.62 | 42.75 | 53.74 | — | 10 | — | 47.4 | — | — | — | — | — | — | — | 99 | 41.16 | 46.64 | |
| PyramidKVModel=Llama-3.1-8B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 30.49 | — | 56.11 | 57.33 | 47.96 | 32.97 | 28.07 | 24.9 | 22.37 | 71 | 91.68 | 42.53 | 53.46 | — | 10.33 | — | 47.38 | — | — | — | — | — | — | — | 99.5 | 42.36 | 47.02 | |
| Full CacheBackbone=Llama-3.1-8B-Instruct, Cache Budget=Full2026.05 | 30.46 | 47.29 | 55.03 | 58.65 | 51.61 | 33.03 | 35.03 | 24.97 | 27.04 | 29 | 84.81 | 39.29 | 51.43 | 44.19 | 10.15 | 100 | 45.12 | — | — | — | — | — | — | — | — | — | — | |
| RDKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 30.41 | 44.54 | 55.82 | 57.14 | 49.25 | 31.77 | 33.14 | 25.19 | 26.92 | 72 | 91.84 | 43.9 | 63.56 | 57.01 | 9.08 | 100 | 49.47 | — | — | — | — | — | — | — | — | — | — | |
| MorphKVModel=Llama-3.1-8B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 30.35 | — | 54.14 | 57.15 | 48.3 | 30.09 | 28.41 | 24.26 | 23.17 | 52.5 | 92.36 | 42.51 | 54.63 | — | 10.12 | — | 45.89 | — | — | — | — | — | — | — | 99.5 | 38.96 | 47.76 | |
| SnapKVModel=Llama-3.1-8B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 30.28 | — | 55.44 | 58.61 | 47.17 | 31.53 | 28.69 | 24.6 | 23.19 | 68.5 | 91.43 | 42.17 | 55.42 | — | 10.25 | — | 47.22 | — | — | — | — | — | — | — | 99.5 | 40.03 | 48.75 | |
| RDKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=512L2026.05 | 30.24 | 45.67 | 55.58 | 57.06 | 48.95 | 31.67 | 30.68 | 25.06 | 26.46 | 72 | 91.83 | 43.15 | 63.36 | 57.02 | 8.83 | 100 | 49.22 | — | — | — | — | — | — | — | — | — | — | |
| Snap+ZipBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 30.23 | 40.65 | 54.13 | 56.06 | 46.23 | 31.45 | 22.84 | 24.5 | 24.28 | 69 | 91.76 | 44.15 | 60.9 | 56.38 | 8.61 | 99 | 47.51 | — | — | — | — | — | — | — | — | — | — | |
| Full AttentionModel=Llama-3.1-8B, Density=100%, Eviction regime=Prefill-only eviction2026.06 | 29.91 | — | 56.16 | 56.96 | 47.75 | 32.77 | 34.7 | 25.39 | 26.76 | 73 | 92.14 | 43.27 | 55.2 | — | 9.66 | — | 48.5 | — | — | — | — | — | — | — | 99.5 | 45.06 | 47.8 | |
| Full AttentionModel=Llama-3.1-8B, Density=100%, Eviction regime=Prefill + decode eviction2026.06 | 29.91 | — | 56.16 | 56.96 | 47.75 | 32.77 | 34.7 | 25.39 | 26.76 | 73 | 92.14 | 43.27 | 55.2 | — | 9.66 | — | 48.5 | — | — | — | — | — | — | — | 99.5 | 45.06 | 47.8 | |
| H2OModel=Llama-3.1-8B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 29.87 | — | 53.24 | 56.97 | 48.52 | 32.14 | 28.05 | 24.42 | 22.17 | 73 | 91.65 | 43.37 | 52.67 | — | 8.96 | — | 46.79 | — | — | — | — | — | — | — | 99 | 38.78 | 45.88 | |
| Snap+ZipBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=64L2026.05 | 29.85 | 32.53 | 52.16 | 55.24 | 46.72 | 30.6 | 19.96 | 22.88 | 21.26 | 52.5 | 90.5 | 40.03 | 61.01 | 51.97 | 9.01 | 75.5 | 43.23 | — | — | — | — | — | — | — | — | — | — | |
| AdaKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 29.76 | 43.39 | 55.82 | 57.62 | 48.31 | 32.35 | 27.22 | 24.75 | 25.18 | 72 | 91.78 | 42.12 | 64.56 | 58.37 | 8.15 | 100 | 48.84 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=512L2026.05 | 29.68 | 39.51 | 55.94 | 56.94 | 48.87 | 30.56 | 24.31 | 24.07 | 23.6 | 65.5 | 91.46 | 41.44 | 64.5 | 56.58 | 8.57 | 100 | 47.6 | — | — | — | — | — | — | — | — | — | — | |
| RDKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=256L2026.05 | 29.63 | 44.85 | 56.33 | 57.85 | 49.64 | 31.73 | 28.01 | 24.66 | 25.66 | 70 | 91.87 | 41.79 | 63.27 | 56.01 | 8.83 | 99.5 | 48.73 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 29.51 | 43.17 | 56.26 | 57.43 | 49.18 | 32.07 | 27.21 | 24.63 | 25.26 | 69.5 | 91.7 | 42.28 | 64.71 | 58.52 | 8.15 | 100 | 48.72 | — | — | — | — | — | — | — | — | — | — | |
| RDKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=128L2026.05 | 29.45 | 41.34 | 55.55 | 57.39 | 49.38 | 30.89 | 25.55 | 24.59 | 24.29 | 65 | 91.92 | 41.8 | 62.68 | 55.41 | 8.75 | 100 | 47.75 | — | — | — | — | — | — | — | — | — | — | |
| Snap+ZipBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=128L2026.05 | 29.4 | 36.64 | 53.26 | 57.27 | 47.23 | 31.09 | 22.59 | 23.66 | 23.97 | 61 | 92 | 42.17 | 62.24 | 53.57 | 8.22 | 95.5 | 46.24 | — | — | — | — | — | — | — | — | — | — | |
| FullKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=Full2026.05 | 29.38 | 44.82 | 55.46 | 57.62 | 49.22 | 32.8 | 34.58 | 25.34 | 26.9 | 73 | 91.51 | 43.64 | 65.24 | 59.08 | 9.04 | 99.5 | 49.82 | — | — | — | — | — | — | — | — | — | — | |
| Snap+ZipBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=512L2026.05 | 29.32 | 41.46 | 53.15 | 55.47 | 46.77 | 33.26 | 23.64 | 24.88 | 24.89 | 68 | 91.75 | 42.32 | 61.75 | 55.03 | 8.22 | 99.5 | 47.46 | — | — | — | — | — | — | — | — | — | — | |
| Nexus SamplingModel=Llama-3.1-8B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 29.24 | — | 57.83 | 58.03 | 48.82 | 32.68 | 30.07 | 25.24 | 22.65 | 73 | 92.66 | 42.93 | 54.62 | — | 10 | — | 48.01 | — | — | — | — | — | — | — | 99 | 43.47 | 47.87 | |
| Full CacheBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=Full2026.05 | 28.38 | 40.72 | 51.68 | 49 | 36.21 | 27.81 | 34.23 | 25.41 | 26.93 | 55.75 | 84.93 | 20.94 | 46.92 | 53.86 | 5.05 | 98 | 42.86 | — | — | — | — | — | — | — | — | — | — | |
| H2OModel=Qwen3-8B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 28.18 | — | 51.76 | 57.83 | 42.08 | 33.73 | 29.6 | 23.28 | 19.5 | 71.5 | 90.71 | 44.81 | 55.19 | — | 3.5 | — | 46.91 | — | — | — | — | — | — | — | 100 | 43.72 | 55.1 | |
| Snap+ZipBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=256L2026.05 | 28.02 | 41.07 | 54.05 | 57.74 | 48.02 | 31.98 | 24.27 | 23.8 | 24.82 | 66 | 91.67 | 42.07 | 61.72 | 54.18 | 7.9 | 99.5 | 47.3 | — | — | — | — | — | — | — | — | — | — | |
| RDKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=64L2026.05 | 28 | 34.47 | 54.16 | 55.9 | 47.45 | 29.65 | 23.1 | 23.78 | 22.54 | 59.5 | 91.76 | 41.52 | 61.7 | 54.1 | 8.7 | 99.17 | 45.97 | — | — | — | — | — | — | — | — | — | — | |
| ThinKBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=512L2026.05 | 27.96 | 40.16 | 55.25 | 56.85 | 49.6 | 31.16 | 26 | 24.36 | 24.28 | 68 | 92 | 41.79 | 64.08 | 57.3 | 8.63 | 100 | 47.96 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVModel=Qwen3-8B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 27.96 | — | 52.76 | 56.94 | 42.38 | 33.92 | 28.76 | 23.17 | 20.29 | 66.5 | 90.71 | 43.9 | 58.07 | — | 6 | — | 46.97 | — | — | — | — | — | — | — | 100 | 43.55 | 56.6 | |
| AdaKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=512L2026.05 | 27.93 | 40.26 | 54.76 | 56.96 | 48.68 | 31.45 | 25.27 | 24.09 | 23.71 | 69.5 | 91.96 | 41.77 | 64.89 | 57.66 | 8.65 | 99.5 | 47.94 | — | — | — | — | — | — | — | — | — | — | |
| FullKVBackbone=Qwen3-30B-A3B-Instruct-2507, KV Cache Budget=20% Cache2026.05 | 27.71 | 40.09 | 53.83 | 66.99 | 62.1 | 32.69 | 30.56 | 22.24 | 24.25 | 76 | 88.82 | 48.17 | 74.62 | 70.8 | 14 | 100 | 52.05 | — | — | — | — | — | — | — | — | — | — | |
| Full AttentionModel=Qwen3-8B, Density=100%, Eviction regime=Prefill-only eviction2026.06 | 27.64 | — | 52.94 | 57.94 | 42.62 | 34.33 | 33.58 | 23.85 | 24.82 | 71.5 | 90.71 | 44.17 | 57.39 | — | 4.5 | — | 48.16 | — | — | — | — | — | — | — | 100 | 47.91 | 56.67 | |
| Full AttentionModel=Qwen3-8B, Density=100%, Eviction regime=Prefill + decode eviction2026.06 | 27.64 | — | 52.94 | 57.94 | 42.62 | 34.33 | 33.58 | 23.85 | 24.82 | 71.5 | 90.71 | 44.17 | 57.39 | — | 4.5 | — | 48.16 | — | — | — | — | — | — | — | 100 | 47.91 | 56.67 | |
| AdaKVBackbone=Qwen3-30B-A3B-Instruct-2507, KV Cache Budget=20% Cache2026.05 | 27.54 | 31.19 | 30.39 | 48 | 39.67 | 18.38 | 25.81 | 18.58 | 19.2 | 45 | 75.58 | 31.11 | 36.81 | 41.78 | 11.09 | 53 | 34.57 | — | — | — | — | — | — | — | — | — | — | |
| Nexus SamplingModel=Qwen3-8B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 27.44 | — | 52.52 | 57.68 | 42.49 | 33.73 | 31.73 | 23.89 | 22.11 | 71 | 91.36 | 43.85 | 56.93 | — | 5 | — | 47.45 | — | — | — | — | — | — | — | 99.5 | 43.72 | 56.23 | |
| ThinKBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=1024L2026.05 | 27.34 | 42.48 | 55.59 | 57.78 | 49.17 | 32.36 | 28.23 | 24.82 | 25.85 | 71.5 | 92.5 | 42.25 | 64.44 | 58.52 | 8.27 | 100 | 48.82 | — | — | — | — | — | — | — | — | — | — | |
| MorphKVModel=Qwen3-8B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 27.26 | — | 50.66 | 55.96 | 40.15 | 31.51 | 29.44 | 23.01 | 20.88 | 53.5 | 90.8 | 43.64 | 57.36 | — | 4.5 | — | 45.34 | — | — | — | — | — | — | — | 100 | 40.45 | 56.32 | |
| Nexus SamplingModel=Qwen3-8B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 26.96 | — | 52.17 | 59.21 | 42.38 | 34.11 | 30.8 | 23.31 | 20.6 | 70 | 91.36 | 43.1 | 56.79 | — | 5 | — | 47.25 | — | — | — | — | — | — | — | 100 | 43.95 | 56.29 | |
| SnapKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=256L2026.05 | 26.9 | 31.27 | 53.45 | 56.8 | 47.97 | 29.52 | 21.99 | 24.34 | 21.31 | 55 | 91.41 | 40.75 | 62.2 | 55.15 | 8.61 | 99 | 45.35 | — | — | — | — | — | — | — | — | — | — | |
| ThinKBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=256L2026.05 | 26.83 | 34.14 | 52.82 | 56.77 | 49.14 | 29.08 | 23.63 | 23.47 | 22.87 | 61.5 | 91.51 | 41.35 | 62.82 | 55.44 | 8.6 | 99.5 | 46.22 | — | — | — | — | — | — | — | — | — | — | |
| FullKVBackbone=Mistral-7B-Instruct-v0.2, KV Budget=Full2026.02 | 26.63 | 32.99 | 49.34 | 42.77 | 27.35 | 18.77 | 32.87 | 24.24 | 27.1 | 71 | 86.23 | 42.96 | 56.93 | 54.49 | 2.75 | 86.98 | 42.71 | — | — | — | — | — | — | — | — | — | — | |
| CriticalKV + GRKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 26.51 | 25.17 | 27.41 | 50.16 | 28.42 | 21.01 | 26.55 | 21.28 | 21.81 | 38.25 | 81.16 | 39.15 | 51.2 | 46.24 | 10 | 71 | 36.58 | — | — | — | — | — | — | 14 | — | — | — | |
| AdaKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=128L2026.05 | 26.35 | 23.5 | 51.59 | 57.11 | 46.38 | 28.36 | 19.68 | 23.09 | 19.58 | 49.5 | 89.92 | 40.25 | 60.42 | 54.59 | 9 | 99 | 43.64 | — | — | — | — | — | — | — | — | — | — | |
| CompilerKVBackbone=Mistral-7B-Instruct-v0.2, KV Budget=512 tokens2026.02 | 26.06 | 29.51 | 49.29 | 41.46 | 27.06 | 18.15 | 25.16 | 24.19 | 25.13 | 70.5 | 86.18 | 42.91 | 55.13 | 51.37 | 3.29 | 84.02 | 41.21 | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVModel=Qwen3-8B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 26.02 | — | 48.48 | 55.25 | 39.56 | 31.78 | 26.87 | 22.29 | 18.44 | 68 | 90.55 | 43.4 | 55.24 | — | 6.5 | — | 45.14 | — | — | — | — | — | — | — | 100 | 36.61 | 53.29 | |
| DynamicKVBackbone=Mistral-7B-Instruct-v0.2, KV Budget=512 tokens2026.02 | 25.63 | 29.11 | 48.41 | 39.8 | 26.62 | 16.72 | 24.73 | 23.42 | 24.83 | 70.5 | 86.74 | 43.01 | 55.4 | 52.45 | 3.2 | 83.57 | 40.88 | — | — | — | — | — | — | — | — | — | — | |
| ThinKBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=128L2026.05 | 25.4 | 30.67 | 49.96 | 55.71 | 46.76 | 28.47 | 21.44 | 23.35 | 20.83 | 50 | 89.87 | 40.97 | 60.18 | 53.22 | 9 | 99 | 44.05 | — | — | — | — | — | — | — | — | — | — | |
| CriticalKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 25.23 | 24.57 | 26.37 | 49.58 | 27.59 | 20.64 | 26.39 | 21.07 | 21.25 | 37.25 | 80.08 | 39.44 | 51.33 | 45.69 | 9.04 | 70.5 | 36 | — | — | — | — | — | — | 0 | — | — | — | |
| AdaKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=256L2026.05 | 25.22 | 32.02 | 52.51 | 56.82 | 47.65 | 30.71 | 22.49 | 23.86 | 21.78 | 61 | 91.27 | 40.92 | 63.42 | 56.58 | 8.63 | 99.75 | 45.91 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=64L2026.05 | 25.2 | 21.05 | 39.35 | 51.69 | 45.25 | 26.57 | 15.06 | 21.24 | 14.6 | 39 | 81.43 | 36.62 | 54.74 | 47.41 | 9 | 95 | 38.95 | — | — | — | — | — | — | — | — | — | — | |
| FullKVBackbone=LLaMA-3-8B-Instruct, KV Budget=Full2026.02 | 25.16 | 31.81 | 39.59 | 43.09 | 36.15 | 21.77 | 28.62 | 23.34 | 26.33 | 75 | 90.5 | 42.36 | 59.04 | 53.93 | 5.2 | 69.25 | 41.95 | — | — | — | — | — | — | — | — | — | — | |
| FullKVBackbone=Qwen2-7B-Instruct, KV Budget=Full2026.02 | 25.14 | 42.35 | 45.04 | 14.8 | 14.13 | 9.23 | 36.35 | 23.79 | 26.51 | 76.5 | 89.16 | 45.23 | 60.3 | 60.78 | 6.5 | 75.5 | 40.71 | — | — | — | — | — | — | — | — | — | — | |
| CompilerKVBackbone=Qwen2-7B-Instruct, KV Budget=512 tokens2026.02 | 25.05 | 41 | 44.99 | 14.75 | 14.04 | 8.5 | 25.96 | 22.7 | 24.16 | 75.5 | 89.09 | 43.38 | 60.76 | 60.1 | 8 | 74 | 39.5 | — | — | — | — | — | — | — | — | — | — | |
| CompilerKVBackbone=LLaMA-3-8B-Instruct, KV Budget=512 tokens2026.02 | 24.97 | 25.11 | 37.01 | 43.04 | 35.69 | 20.86 | 23.66 | 22.93 | 25.4 | 72.5 | 90.21 | 40.67 | 61.47 | 57 | 5.64 | 69.5 | 40.98 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Mistral-7B-Instruct-v0.2, KV Budget=512 tokens2026.02 | 24.96 | 27.97 | 49.04 | 39.93 | 25.18 | 17.64 | 24.14 | 23.69 | 24.41 | 67.5 | 86.09 | 41.11 | 56.73 | 53.11 | 2.86 | 86.98 | 40.71 | — | — | — | — | — | — | — | — | — | — | |
| ThinKBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=64L2026.05 | 24.93 | 23.54 | 45.55 | 53.54 | 44.17 | 27.79 | 18.38 | 22.77 | 18.1 | 42 | 84.35 | 38.5 | 55.81 | 49.22 | 9 | 98.5 | 41.01 | — | — | — | — | — | — | — | — | — | — | |
| DynamicKVBackbone=LLaMA-3-8B-Instruct, KV Budget=512 tokens2026.02 | 24.8 | 24.62 | 36.69 | 44.13 | 33.25 | 20.82 | 23 | 22.54 | 24.12 | 72.5 | 90.39 | 40.76 | 61.21 | 56.91 | 5.78 | 69.5 | 40.69 | — | — | — | — | — | — | — | — | — | — | |
| DynamicKVBackbone=Qwen2-7B-Instruct, KV Budget=512 tokens2026.02 | 24.66 | 40.44 | 45.3 | 15.42 | 13.89 | 8.46 | 25.51 | 22.77 | 22.92 | 74 | 89.27 | 43.18 | 60.38 | 59.32 | 7 | 74 | 39.16 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=LLaMA-3-8B-Instruct, KV Budget=512 tokens2026.02 | 24.62 | 22.78 | 37.88 | 42.96 | 34.82 | 20.25 | 22.63 | 22.54 | 23.93 | 70 | 90.39 | 40.3 | 60.27 | 55.85 | 5.74 | 69.5 | 40.28 | — | — | — | — | — | — | — | — | — | — | |
| AdaKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=64L2026.05 | 24.62 | 19.62 | 39.41 | 50.93 | 42.13 | 26.1 | 16.08 | 22.17 | 15.71 | 39.5 | 85.62 | 37.83 | 56.57 | 50.7 | 9 | 97.5 | 39.59 | — | — | — | — | — | — | — | — | — | — | |
| ZigZagKVBackbone=LLaMA-3-8B-Instruct, KV Budget=512 tokens2026.02 | 24.54 | 23.18 | 36.98 | 42.61 | 33.24 | 20.51 | 23.04 | 22.8 | 24.17 | 71.4 | 90.41 | 40.43 | 59.72 | 55.31 | 5.81 | 69.5 | 40.66 | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=LLaMA-3-8B-Instruct, KV Budget=512 tokens2026.02 | 24.48 | 23.51 | 36.24 | 42.33 | 31.95 | 20.73 | 23.37 | 23.01 | 24.37 | 72.5 | 90.43 | 40.54 | 59.27 | 54.87 | 5.88 | 69.5 | 40.19 | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVBackbone=Qwen2-7B-Instruct, KV Budget=512 tokens2026.02 | 24.47 | 37.6 | 43.51 | 14.48 | 12.83 | 8.99 | 23.59 | 22.3 | 22.41 | 74 | 89.21 | 43.4 | 57.67 | 56.14 | 6.5 | 74 | 38.19 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=LLaMA-3.1-8B-Instruct, Cache budget (Btotal)=128L2026.05 | 24.41 | 24.19 | 48.91 | 55.98 | 45.51 | 28.41 | 18.64 | 22.76 | 18.72 | 47 | 88.91 | 39.41 | 59.14 | 52.18 | 9 | 98 | 42.57 | — | — | — | — | — | — | — | — | — | — | |
| SnapKV + GRKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 24.22 | 21.84 | 25.39 | 47.97 | 27.32 | 20.08 | 25.41 | 20.75 | 21.07 | 35.5 | 81.41 | 38.97 | 49.69 | 45.14 | 9 | 59.5 | 34.58 | — | — | — | — | — | — | 14 | — | — | — | |
| ZigZagKVBackbone=Qwen2-7B-Instruct, KV Budget=512 tokens2026.02 | 24.19 | 38.01 | 44.04 | 14.98 | 13.64 | 9.05 | 24.03 | 22.34 | 22.76 | 72.5 | 89.26 | 43.36 | 58.12 | 58.21 | 5.8 | 73 | 38.8 | — | — | — | — | — | — | — | — | — | — | |
| ZigZagKVBackbone=Mistral-7B-Instruct-v0.2, KV Budget=512 tokens2026.02 | 24.18 | 28.42 | 48.86 | 40.52 | 25.43 | 16.93 | 24.49 | 23.59 | 24.45 | 68.5 | 86.15 | 41.91 | 56.03 | 52.32 | 3.22 | 84.14 | 40.83 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 24.15 | 20.36 | 25.41 | 46.87 | 26.67 | 18.95 | 24.79 | 20.37 | 20.76 | 34.5 | 80.08 | 38.65 | 50.05 | 44.72 | 8 | 59 | 33.96 | — | — | — | — | — | — | 0 | — | — | — | |
| SnapKVBackbone=Qwen2-7B-Instruct, KV Budget=512 tokens2026.02 | 23.86 | 38.5 | 44.68 | 15.6 | 14.62 | 9.13 | 24.56 | 22.39 | 23.18 | 70 | 89.31 | 43.32 | 58.68 | 60.74 | 5 | 72 | 38.47 | — | — | — | — | — | — | — | — | — | — | |
| SnapKV + CaMBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 23.86 | 20.15 | 26.15 | 48.79 | 26.12 | 20.2 | 23.93 | 20.35 | 20.33 | 24.5 | 79.88 | 32.16 | 43.35 | 42.48 | 4.53 | 59.5 | 32.27 | — | — | — | — | — | — | 4 | — | — | — | |
| PyramidKVBackbone=Mistral-7B-Instruct-v0.2, KV Budget=512 tokens2026.02 | 23.55 | 28.79 | 48.71 | 41 | 25.64 | 16.35 | 24.79 | 23.52 | 24.49 | 69.5 | 86.2 | 42.58 | 55.45 | 51.67 | 3.53 | 81.81 | 40.47 | — | — | — | — | — | — | — | — | — | — | |
| CriticalKV + CaMBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 23.47 | 21.59 | 28.89 | 50.96 | 28.93 | 22.1 | 24.96 | 21.14 | 21.11 | 24.5 | 79.88 | 31.72 | 42.53 | 42.33 | 6.75 | 77.5 | 34.27 | — | — | — | — | — | — | 6 | — | — | — | |
| H2OBackbone=Qwen2-7B-Instruct, KV Budget=512 tokens2026.02 | 22.88 | 34.28 | 41.5 | 13.3 | 14.6 | 8.31 | 23.69 | 22.07 | 22.72 | 39.5 | 88.75 | 43.91 | 58.83 | 57.83 | 8 | 72 | 35.76 | — | — | — | — | — | — | — | — | — | — | |
| H2OBackbone=LLaMA-3-8B-Instruct, KV Budget=512 tokens2026.02 | 22.84 | 16.8 | 32.36 | 41.5 | 34.07 | 19.33 | 22.28 | 22.88 | 23.69 | 41 | 90.46 | 40.22 | 57.52 | 55.43 | 5.54 | 69.5 | 37.21 | — | — | — | — | — | — | — | — | — | — | |
| FullKVBackbone=InternLM-2.5-7B-Chat-1M, KV Budget=Full2026.02 | 22.47 | 27.58 | 39.98 | 40.96 | 33.52 | 26.61 | 33.01 | 25.18 | 26.28 | 72.5 | 86.76 | 39.84 | 55.86 | 57.9 | 2.92 | 100 | 43.21 | — | — | — | — | — | — | — | — | — | — | |
| H2OBackbone=Mistral-7B-Instruct-v0.2, KV Budget=512 tokens2026.02 | 22.23 | 25.75 | 44.09 | 32.76 | 22.88 | 14.96 | 23.53 | 22.96 | 24.53 | 41.5 | 85.53 | 41.54 | 55.11 | 50.81 | 3.39 | 86.2 | 37.36 | — | — | — | — | — | — | — | — | — | — | |
| Nexus SamplingModel=Llama-3.2-1B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 21.59 | — | 41.16 | 34.41 | 30.97 | 17.62 | 24.63 | 21.01 | 20.81 | 62 | 78.88 | 38.73 | 30.46 | — | 4.64 | — | 30.29 | — | — | — | — | — | — | — | 5 | 15.18 | 37.48 | |
| SnapKVModel=Llama-3.2-1B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 21.58 | — | 39.04 | 35.48 | 30.45 | 17.15 | 22.92 | 21.18 | 19.61 | 58 | 79.31 | 37.4 | 30.75 | — | 3.67 | — | 29.53 | — | — | — | — | — | — | — | 5 | 15.06 | 35.86 | |
| Nexus SamplingModel=Llama-3.2-1B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 20.75 | — | 41.87 | 34.52 | 31.86 | 17.53 | 24.58 | 21.74 | 20.91 | 58.5 | 79.95 | 37.71 | 30.79 | — | 5.67 | — | 30.18 | — | — | — | — | — | — | — | 5 | 15.3 | 36.23 | |
| H2OModel=Llama-3.2-1B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 20.65 | — | 39.66 | 35.45 | 30.8 | 17.63 | 23.38 | 21.16 | 21 | 61.5 | 78.5 | 38.92 | 28.62 | — | 2.67 | — | 29.6 | — | — | — | — | — | — | — | 4.5 | 14.62 | 34.6 | |
| SnapKV + AsymKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=10%2026.05 | 20.48 | 17.13 | 27.35 | 40.24 | 23.38 | 16.02 | 25.76 | 20.5 | 22.96 | 42.25 | 81.17 | 18.12 | 44.69 | 48.78 | 4.83 | 32.25 | 30.37 | — | — | — | — | — | — | 8 | — | — | — | |
| StreamingKVBackbone=Qwen2-7B-Instruct, KV Budget=512 tokens2026.02 | 20.47 | 26.97 | 32.64 | 14.31 | 14.39 | 6.82 | 25.7 | 19.31 | 24.88 | 66 | 76.56 | 32.11 | 46.58 | 44.2 | 8 | 15.5 | 29.65 | — | — | — | — | — | — | — | — | — | — | |
| Full AttentionModel=Llama-3.2-1B, Density=100%, Eviction regime=Prefill-only eviction2026.06 | 20.4 | — | 43.35 | 35.47 | 31.23 | 18.45 | 29.65 | 21.84 | 25.88 | 61.5 | 78.54 | 39.77 | 29.83 | — | 3.67 | — | 31.03 | — | — | — | — | — | — | — | 4.5 | 16.35 | 36.04 | |
| Full AttentionModel=Llama-3.2-1B, Density=100%, Eviction regime=Prefill + decode eviction2026.06 | 20.4 | — | 43.35 | 35.47 | 31.23 | 18.45 | 29.65 | 21.84 | 25.88 | 61.5 | 78.54 | 39.77 | 29.83 | — | 3.67 | — | 31.03 | — | — | — | — | — | — | — | 4.5 | 16.35 | 36.04 | |
| CriticalKV + AsymKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=10%2026.05 | 20.38 | 19.65 | 28.51 | 36.83 | 23.15 | 14.12 | 25.9 | 20.88 | 23.45 | 45.25 | 80.67 | 16.46 | 44.8 | 48.98 | 2.83 | 26 | 29.87 | — | — | — | — | — | — | 6 | — | — | — | |
| CriticalKV + AsymKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 20.17 | 20.77 | 27.41 | 35.27 | 28.42 | 11.94 | 24.3 | 20.97 | 23.57 | 14 | 83.19 | 21.63 | 51.58 | 45.84 | 1.52 | 14 | 27.79 | — | — | — | — | — | — | 6 | — | — | — | |
| SnapKV + AsymKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 19.91 | 22.35 | 27.31 | 40.18 | 25.05 | 13.66 | 24.92 | 21.13 | 23.25 | 19 | 87.94 | 20.59 | 48.28 | 46.12 | 2.83 | 21 | 28.97 | — | — | — | — | — | — | 7 | — | — | — | |
| CriticalKV + D2OBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 19.85 | 13.06 | 21.28 | 37.67 | 18.99 | 14.05 | 22.68 | 18.41 | 19.14 | 21.5 | 77.71 | 36.38 | 45.43 | 45.34 | 9 | 24 | 27.78 | — | — | — | — | — | — | 0 | — | — | — | |
| PyramidKVModel=Llama-3.2-1B, Density=20%, Eviction regime=Prefill-only eviction2026.06 | 19.63 | — | 39.91 | 34.89 | 28.4 | 16.19 | 20.27 | 20.8 | 16.92 | 58 | 78.45 | 37.37 | 28.62 | — | 3 | — | 28.57 | — | — | — | — | — | — | — | 5 | 14.47 | 35.13 | |
| MorphKVModel=Llama-3.2-1B, Density=20%, Eviction regime=Prefill + decode eviction2026.06 | 19.56 | — | 34.95 | 33.64 | 28.89 | 16.78 | 23.58 | 21.17 | 21.51 | 47 | 79.5 | 36.77 | 28.69 | — | 3.17 | — | 28.08 | — | — | — | — | — | — | — | 4.5 | 14.5 | 35.12 | |
| StreamingKVBackbone=Mistral-7B-Instruct-v0.2, KV Budget=512 tokens2026.02 | 19.05 | 17.21 | 36.82 | 30.64 | 21.84 | 10.56 | 24.47 | 19.84 | 25.48 | 62 | 72.82 | 29.49 | 46.15 | 42.55 | 2.71 | 19.25 | 30.06 | — | — | — | — | — | — | — | — | — | — | |
| StreamingKVBackbone=LLaMA-3-8B-Instruct, KV Budget=512 tokens2026.02 | 19.03 | 12.78 | 28.67 | 37.83 | 29.97 | 16.55 | 20.3 | 20.94 | 24.56 | 61 | 75.43 | 30.82 | 51.93 | 49.98 | 5.86 | 69.5 | 34.7 | — | — | — | — | — | — | — | — | — | — | |
| CriticalKV + GRKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=10%2026.05 | 19 | 15.55 | 35.64 | 37.68 | 25.78 | 18.28 | 26.18 | 20.98 | 21.58 | 42.25 | 86.84 | 27.5 | 50.16 | 51.42 | 3.93 | 66 | 34.3 | — | — | — | — | — | — | 14 | — | — | — | |
| CriticalKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=10%2026.05 | 18.99 | 14.38 | 34.95 | 36.34 | 26.12 | 18.43 | 26.09 | 20.66 | 21.14 | 40.75 | 86.31 | 27.45 | 49.02 | 51.1 | 3.33 | 64 | 33.69 | — | — | — | — | — | — | 0 | — | — | — | |
| CompilerKVBackbone=InternLM-2.5-7B-Chat-1M, KV Budget=512 tokens2026.02 | 18.47 | 24.22 | 38.37 | 33.49 | 21.25 | 23.05 | 24.41 | 24.05 | 23.68 | 75 | 84.31 | 36.11 | 51.16 | 52.88 | 1 | 95.5 | 39.18 | — | — | — | — | — | — | — | — | — | — | |
| SnapKV + GRKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=10%2026.05 | 18.43 | 13.38 | 30.79 | 37.23 | 25.71 | 15.58 | 25.38 | 21.02 | 21.11 | 40 | 88.27 | 29.82 | 51.32 | 53.13 | 3.39 | 65.5 | 33.75 | — | — | — | — | — | — | 12 | — | — | — | |
| DynamicKVBackbone=InternLM-2.5-7B-Chat-1M, KV Budget=512 tokens2026.02 | 17.77 | 23.87 | 37.74 | 32.98 | 21.13 | 23.21 | 19.13 | 23.49 | 22.48 | 75 | 84.89 | 36.7 | 50.7 | 51.08 | 0.91 | 95.5 | 38.54 | — | — | — | — | — | — | — | — | — | — | |
| SnapKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=10%2026.05 | 17.72 | 12.61 | 30.07 | 34.28 | 25.68 | 15.52 | 25.42 | 21.1 | 20.77 | 37.75 | 87.4 | 28.78 | 50.63 | 52.32 | 3.92 | 66 | 33.12 | — | — | — | — | — | — | 0 | — | — | — | |
| SnapKV + D2OBackbone=Llama-3.1-8B-Instruct, Cache Budget=10%2026.05 | 17.63 | 13.24 | 18.53 | 35.8 | 17.99 | 12.44 | 22.05 | 17.73 | 18.83 | 21.5 | 80.71 | 35.97 | 46.5 | 45.68 | 5.5 | 29 | 27.44 | — | — | — | — | — | — | 2 | — | — | — | |
| PyramidKVBackbone=InternLM-2.5-7B-Chat-1M, KV Budget=512 tokens2026.02 | 17.62 | 21.08 | 37.52 | 32.21 | 21.31 | 22.03 | 19.37 | 24.06 | 22.22 | 73 | 83.94 | 34.61 | 50.45 | 49.72 | 1.05 | 95.5 | 37.86 | — | — | — | — | — | — | — | — | — | — |