Long-context Language Modeling on LongBench (Aggregated Metrics)
58.4Average ScoreOriginal
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OriginalModel=Qwen3-14B-Base2026.04 | 58.4 | 43.7 | 43.5 | 27 | 72.1 | 66.7 | 66.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 89.2 | — | — | — | — | — | — | |
| OriginalModel=Qwen3-8B-Base2026.04 | 57.1 | 45.5 | 41.9 | 26.8 | 70.3 | 63 | 66.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 85.8 | — | — | — | — | — | — | |
| OriginalModel=Qwen3-30B-A3B-Base2026.04 | 54.9 | 24.6 | 39.8 | 27.6 | 70 | 64.3 | 67.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 90.1 | — | — | — | — | — | — | |
| FullKVBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=Full2026.05 | 54.08 | — | — | — | — | — | — | 32.21 | 49.59 | 51.88 | 65.26 | 65.51 | 41.37 | 34 | 24.35 | 24.74 | 78 | 91.76 | 47.87 | 21.5 | 99.5 | 66.36 | 71.42 | — | — | — | — | — | — | — | |
| RDKVBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=512L2026.05 | 53.82 | — | — | — | — | — | — | 32.65 | 49.51 | 51.68 | 64.77 | 64.95 | 40.41 | 31.78 | 24.21 | 23.94 | 78 | 92.06 | 47.91 | 21.5 | 99.5 | 66.34 | 71.9 | — | — | — | — | — | — | — | |
| Snap+ZipBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=512L2026.05 | 53.54 | — | — | — | — | — | — | 32.5 | 50.24 | 51.8 | 64.21 | 64.33 | 41 | 31.63 | 23.87 | 24.23 | 78 | 92.61 | 46.28 | 21.5 | 99.5 | 65.03 | 69.85 | — | — | — | — | — | — | — | |
| AdaKVBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=512L2026.05 | 52.72 | — | — | — | — | — | — | 33.78 | 45.16 | 50.82 | 63.96 | 64.8 | 40.83 | 27.41 | 23.02 | 22.29 | 76.5 | 91.89 | 45.6 | 22 | 99.5 | 65.48 | 70.48 | — | — | — | — | — | — | — | |
| SnapKVBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=512L2026.05 | 52.49 | — | — | — | — | — | — | 32.5 | 45.81 | 51.23 | 64.56 | 64.39 | 40.55 | 26.89 | 23.24 | 22.07 | 75.5 | 92.16 | 45.03 | 21.5 | 99.5 | 65.04 | 69.95 | — | — | — | — | — | — | — | |
| RDKVBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=128L2026.05 | 52.3 | — | — | — | — | — | — | 31.76 | 44.79 | 49.57 | 63.86 | 64.42 | 39.72 | 26.93 | 23.49 | 22.1 | 76 | 92.16 | 45.15 | 22 | 99.5 | 65.48 | 69.8 | — | — | — | — | — | — | — | |
| ThinKBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=512L2026.05 | 52.05 | — | — | — | — | — | — | 31.03 | 43.89 | 50.81 | 64.59 | 64.58 | 39.35 | 26.33 | 23.42 | 21.92 | 73.5 | 92.19 | 45.76 | 21 | 99.17 | 65.34 | 69.89 | — | — | — | — | — | — | — | |
| Snap+ZipBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=128L2026.05 | 51.91 | — | — | — | — | — | — | 30.6 | 46.38 | 50.5 | 65.47 | 64.97 | 40.35 | 26.64 | 22.3 | 22.14 | 73 | 91.86 | 45.29 | 19.5 | 99 | 64.03 | 68.52 | — | — | — | — | — | — | — | |
| BOSCHModel=Qwen3-14B-Base, Continual Pretraining Tokens=2.5B, SWA ratio=0.752026.04 | 51.5 | 42.6 | 36.2 | 17.6 | 68.8 | 55.2 | 56.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.5 | — | — | — | — | — | — | |
| BOSCHModel=Qwen3-8B-Base, Continual Pretraining Tokens=2.0B, SWA ratio=0.752026.04 | 50.6 | 46.3 | 37.6 | 24.7 | 65.7 | 41.3 | 54.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.4 | — | — | — | — | — | — | |
| BOSCHModel=Qwen3-8B-Base, Continual Pretraining Tokens=2.5B, SWA ratio=0.752026.04 | 50.5 | 46 | 38.3 | 24.5 | 65.4 | 41 | 54.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.5 | — | — | — | — | — | — | |
| BOSCHModel=Qwen3-8B-Base, Continual Pretraining Tokens=1.3B, SWA ratio=0.752026.04 | 49.9 | 46.1 | 38.7 | 24.6 | 65.3 | 37.5 | 52.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 84 | — | — | — | — | — | — | |
| PolarQuant33Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=3.252025.02 | 49.53 | — | — | — | — | — | — | 32.49 | 46.72 | 56.56 | 58.33 | 49.96 | 32.2 | — | — | — | — | — | — | — | — | 63.45 | — | — | — | — | — | — | — | 56.54 | |
| ZipCache4Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=4.252025.02 | 49.45 | — | — | — | — | — | — | 32.26 | 45.97 | 56.77 | 58.67 | 49.5 | 33.03 | — | — | — | — | — | — | — | — | 63.41 | — | — | — | — | — | — | — | 55.97 | |
| PolarQuant44Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=4.252025.02 | 49.39 | — | — | — | — | — | — | 31.36 | 46.78 | 56.72 | 58.54 | 49.47 | 32.23 | — | — | — | — | — | — | — | — | 63.28 | — | — | — | — | — | — | — | 56.73 | |
| KIVI-4Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=4.252025.02 | 49.36 | — | — | — | — | — | — | 31.23 | 47.15 | 57.14 | 58.05 | 49.14 | 32.67 | — | — | — | — | — | — | — | — | 63.05 | — | — | — | — | — | — | — | 56.45 | |
| BOSCHModel=Qwen3-8B-Base, Continual Pretraining Tokens=0.7B, SWA ratio=0.752026.04 | 49.3 | 45.5 | 36.2 | 25.6 | 64.9 | 34.5 | 58.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 79.9 | — | — | — | — | — | — | |
| Bf16Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=162025.02 | 49.26 | — | — | — | — | — | — | 31.38 | 46.65 | 56.81 | 57.85 | 49.46 | 32.63 | — | — | — | — | — | — | — | — | 62.88 | — | — | — | — | — | — | — | 56.43 | |
| Full CacheModel=Llama-3.1-8B, Cache Budget=100%2025.02 | 49.2 | 43.1 | 46.49 | 28.97 | 69.45 | 53.73 | 57.86 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0 | — | |
| Int-4Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=4.252025.02 | 49.05 | — | — | — | — | — | — | 31.76 | 45.49 | 56.47 | 57.67 | 49.52 | 32.82 | — | — | — | — | — | — | — | — | 63.17 | — | — | — | — | — | — | — | 55.46 | |
| AdaKVBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=128L2026.05 | 48.86 | — | — | — | — | — | — | 30.68 | 33.07 | 49.54 | 63.92 | 62.93 | 38.35 | 21.8 | 22.02 | 18.46 | 61.5 | 90.91 | 42.15 | 21.5 | 99.25 | 59.94 | 65.71 | — | — | — | — | — | — | — | |
| KIVI-2Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=3.002025.02 | 48.68 | — | — | — | — | — | — | 31.9 | 45.39 | 54.96 | 58.08 | 49.88 | 32.25 | — | — | — | — | — | — | — | — | 62.03 | — | — | — | — | — | — | — | 54.93 | |
| QJLBackbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=3.132025.02 | 48.61 | — | — | — | — | — | — | 32.41 | 44.75 | 56.18 | 57.34 | 48.5 | 32.07 | — | — | — | — | — | — | — | — | 61.66 | — | — | — | — | — | — | — | 55.99 | |
| CriticalKV (HeadKV)Model=Qwen2.5-32B, Cache Budget=40%, Base Method=HeadKV2025.02 | 48.59 | 40.49 | 55.33 | 26.36 | 69.27 | 55.04 | 46.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0 | — | |
| Full CacheModel=Qwen2.5-32B, Cache Budget=100%2025.02 | 48.58 | 43.23 | 54.03 | 27.4 | 68.97 | 56.25 | 41.93 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0 | — | |
| ZipCache3Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=3.252025.02 | 48.43 | — | — | — | — | — | — | 31.98 | 44.7 | 55.61 | 58.33 | 49.16 | 31.53 | — | — | — | — | — | — | — | — | 61.93 | — | — | — | — | — | — | — | 54.19 | |
| Int-3Backbone=Llama-3.1-8B-Instruct, Context Window=128K, Bits=3.252025.02 | 48.26 | — | — | — | — | — | — | 29.66 | 45.07 | 55.15 | 58.31 | 49.79 | 32.73 | — | — | — | — | — | — | — | — | 60.56 | — | — | — | — | — | — | — | 54.8 | |
| CriticalKV (AdaKV)Model=Llama-3.1-8B, Cache Budget=40%, Base Method=AdaKV2025.02 | 48 | 39.24 | 45.11 | 27.31 | 68.19 | 53.6 | 60.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.4 | — | |
| CapKVBackbone=Qwen3-4B, C.R.=0.252026.04 | 47.86 | — | — | — | — | — | — | — | — | — | 62.05 | 45.16 | 34.63 | 31.57 | 24.72 | 25.35 | 67.5 | 87.35 | 40.55 | 4.5 | — | 63.04 | 59.7 | — | 26.76 | 55.33 | 98 | 39.59 | — | — | |
| HeadKVModel=Qwen2.5-32B, Cache Budget=40%2025.02 | 47.81 | 38.74 | 55.13 | 26.1 | 68.37 | 55.09 | 44.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 1.6 | — | |
| CriticalKV (HeadKV)Model=Llama-3.1-8B, Cache Budget=40%, Base Method=HeadKV2025.02 | 47.79 | 40.61 | 44.33 | 27.24 | 68.06 | 54.04 | 57.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.9 | — | |
| Full CacheModel=Mistral-7B, Cache Budget=100%2025.02 | 47.38 | 38.37 | 39.4 | 28.76 | 70.33 | 52.5 | 61.25 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0 | — | |
| CriticalKV (SnapKV)Model=Llama-3.1-8B, Cache Budget=40%, Base Method=SnapKV2025.02 | 47.29 | 38.27 | 43.17 | 27.15 | 66.87 | 54.22 | 60.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.9 | — | |
| SnapKVBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=128L2026.05 | 47.25 | — | — | — | — | — | — | 28.76 | 32 | 46.33 | 62.37 | 62.21 | 36.6 | 20.94 | 21.38 | 17.9 | 53 | 90.26 | 40.81 | 23 | 97.42 | 59.07 | 63.89 | — | — | — | — | — | — | — | |
| Full Attention2025.08 | 47.25 | — | — | — | — | — | — | 30.11 | 45.4 | 54.85 | 55.64 | 46.4 | 31.28 | 35.17 | 25.26 | 27.15 | — | 91.65 | — | — | — | 65.12 | 58.97 | — | — | — | — | — | — | — | |
| HeadKVModel=Llama-3.1-8B, Cache Budget=40%2025.02 | 47.23 | 39.96 | 43.1 | 27.06 | 65.64 | 52.89 | 61.34 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 4 | — | |
| CriticalKV (SnapKV)Model=Qwen2.5-32B, Cache Budget=40%, Base Method=SnapKV2025.02 | 47.23 | 37.18 | 54.55 | 26.03 | 67.8 | 55.75 | 43.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.8 | — | |
| CriticalKV (AdaKV)Model=Qwen2.5-32B, Cache Budget=40%, Base Method=AdaKV2025.02 | 47.03 | 35.7 | 53.75 | 25.97 | 68.65 | 55.25 | 44.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 3.2 | — | |
| RetroAttentionBudget=0.15, Window size (w)=82025.08 | 46.99 | — | — | — | — | — | — | 30.08 | 45.06 | 54.78 | 55.39 | 46.15 | 31.2 | 34.4 | 25.63 | 26.77 | — | 91.5 | — | — | — | 63.78 | 59.21 | — | — | — | — | — | — | — | |
| QuestBudget=0.152025.08 | 46.97 | — | — | — | — | — | — | 29.99 | 45.35 | 54.27 | 55.5 | 46.29 | 31.27 | 34.54 | 25.36 | 26.64 | — | 91.49 | — | — | — | 64.06 | 58.88 | — | — | — | — | — | — | — | |
| RetroAttentionBudget=0.15, Window size (w)=42025.08 | 46.95 | — | — | — | — | — | — | 30.1 | 44.73 | 54.76 | 55.39 | 46.19 | 31.2 | 34.35 | 25.34 | 26.72 | — | 91.5 | — | — | — | 63.72 | 59.35 | — | — | — | — | — | — | — | |
| QuestBudget=0.102025.08 | 46.9 | — | — | — | — | — | — | 30.03 | 44.39 | 55.38 | 55.42 | 46.44 | 31.24 | 34.15 | 25.64 | 26.37 | — | 91.38 | — | — | — | 64.16 | 58.23 | — | — | — | — | — | — | — | |
| RetroAttentionBudget=0.10, Window size (w)=42025.08 | 46.89 | — | — | — | — | — | — | 30.1 | 44.85 | 54.83 | 55.5 | 45.92 | 31.23 | 34 | 25.45 | 26.9 | — | 91.49 | — | — | — | 63.85 | 58.6 | — | — | — | — | — | — | — | |
| RetroAttentionBudget=0.15, Window size (w)=22025.08 | 46.86 | — | — | — | — | — | — | 29.98 | 44.47 | 54.56 | 55.39 | 46.24 | 31.21 | 34.42 | 25.36 | 26.32 | — | 91.5 | — | — | — | 63.94 | 58.98 | — | — | — | — | — | — | — | |
| RetroAttentionBudget=0.10, Window size (w)=22025.08 | 46.83 | — | — | — | — | — | — | 30.07 | 44.43 | 55.06 | 55.64 | 46 | 31.2 | 33.88 | 25.62 | 26.31 | — | 91.49 | — | — | — | 63.65 | 58.63 | — | — | — | — | — | — | — | |
| RetroAttentionBudget=0.10, Window size (w)=82025.08 | 46.76 | — | — | — | — | — | — | 29.91 | 44.12 | 54.97 | 55.22 | 45.92 | 31.22 | 33.81 | 25.71 | 26.43 | — | 91.49 | — | — | — | 63.66 | 58.64 | — | — | — | — | — | — | — | |
| QWEN3-4BBackbone=Qwen3-4B2026.04 | 46.63 | — | — | — | — | — | — | — | — | — | 60.5 | 46.34 | 35.17 | 31.23 | 24.9 | 25.32 | 48.5 | 87.18 | 39.75 | 6 | — | 61.88 | 55.78 | — | 27.26 | 56.13 | 99 | 41.1 | — | — | |
| EABackbone=Qwen3-4B, C.R.=0.252026.04 | 46.47 | — | — | — | — | — | — | — | — | — | 61.52 | 44.06 | 34.2 | 31.77 | 24.36 | 25.11 | 51.5 | 88.01 | 39.82 | 5.5 | — | 62.96 | 56.23 | — | 26.01 | 55.49 | 98 | 38.9 | — | — | |
| CapKVBackbone=Qwen3-4B, C.R.=0.52026.04 | 46.41 | — | — | — | — | — | — | — | — | — | 58.64 | 43.84 | 35.21 | 30.75 | 24.24 | 24.39 | 61 | 87.93 | 40.3 | 4 | — | 57.23 | 60.79 | — | 25.69 | 52.09 | 98.67 | 37.77 | — | — | |
| ThinKBackbone=Qwen2.5-72B-Instruct (2-GPU TP), Btotal=128L2026.05 | 46.38 | — | — | — | — | — | — | 28.02 | 29.54 | 46.3 | 62.17 | 61.47 | 37.73 | 20.72 | 21.27 | 17.7 | 47 | 90.28 | 40.22 | 21.5 | 96.17 | 58.42 | 63.51 | — | — | — | — | — | — | — | |
| SNAPKVBackbone=Qwen3-4B, C.R.=0.252026.04 | 46.31 | — | — | — | — | — | — | — | — | — | 61.22 | 45.19 | 35.5 | 31.07 | 24.78 | 25.16 | 47.5 | 87.52 | 40.39 | 6 | — | 63.01 | 56.76 | — | 26.5 | 53.48 | 99 | 37.86 | — | — | |
| QuestBudget=0.052025.08 | 46.3 | — | — | — | — | — | — | 30.29 | 42.54 | 54.53 | 54.4 | 45.81 | 31.03 | 33.81 | 25.1 | 26.26 | — | 91.68 | — | — | — | 62.97 | 57.16 | — | — | — | — | — | — | — | |
| RetroAttentionBudget=0.05, Window size (w)=22025.08 | 46.26 | — | — | — | — | — | — | 29.11 | 43.52 | 53.89 | 54.44 | 44.57 | 31.46 | 33.43 | 25.37 | 26.2 | — | 91.66 | — | — | — | 63.65 | 57.87 | — | — | — | — | — | — | — | |
| RetroAttentionBudget=0.05, Window size (w)=42025.08 | 46.25 | — | — | — | — | — | — | 29.68 | 43.67 | 53.08 | 54.42 | 44.93 | 30.95 | 33.29 | 25.83 | 26.49 | — | 91.68 | — | — | — | 63.58 | 57.43 | — | — | — | — | — | — | — | |
| AdaKVModel=Llama-3.1-8B, Cache Budget=40%2025.02 | 46.24 | 36.63 | 41.36 | 26.66 | 66.43 | 53 | 60.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6 | — | |
| CriticalKV (AdaKV)Model=Mistral-7B, Cache Budget=40%, Base Method=AdaKV2025.02 | 46.23 | 36.06 | 38 | 27.06 | 69.79 | 50.84 | 62.68 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.4 | — | |
| RetroAttentionBudget=0.05, Window size (w)=82025.08 | 46.18 | — | — | — | — | — | — | 29.74 | 42.96 | 53.4 | 54.33 | 44.9 | 31.29 | 33.3 | 25.52 | 26.54 | — | 91.68 | — | — | — | 63.47 | 57.06 | — | — | — | — | — | — | — | |
| CriticalKV (HeadKV)Model=Mistral-7B, Cache Budget=40%, Base Method=HeadKV2025.02 | 46.1 | 36.21 | 37.66 | 27.72 | 69.78 | 50.2 | 61.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 2.7 | — | |
| AdaKVModel=Qwen2.5-32B, Cache Budget=40%2025.02 | 46.03 | 32.37 | 52.75 | 24.95 | 68.18 | 54.63 | 46.21 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 5.3 | — | |
| EABackbone=Qwen3-4B, C.R.=0.52026.04 | 45.67 | — | — | — | — | — | — | — | — | — | 59.75 | 43.73 | 34.91 | 31.25 | 23.92 | 25 | 62.5 | 87.95 | 39.66 | 5.5 | — | 62.62 | 57.3 | — | 27.18 | 50.03 | 82.5 | 36.97 | — | — | |
| SnapKVModel=Llama-3.1-8B, Cache Budget=40%2025.02 | 45.52 | 35.27 | 40.5 | 26.11 | 65.1 | 53.17 | 60.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.5 | — | |
| SnapKVModel=Qwen2.5-32B, Cache Budget=40%2025.02 | 45.47 | 32.22 | 50.9 | 25.19 | 67.43 | 55.25 | 44.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.4 | — | |
| CriticalKV (HeadKV)Model=Qwen2.5-32B, Cache Budget=20%, Base Method=HeadKV2025.02 | 45.43 | 32.07 | 50.6 | 24.34 | 68.53 | 53.88 | 46.27 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.5 | — | |
| OriginalModel=Qwen3-1.7B-Base2026.04 | 45.4 | 39.8 | 30.9 | 26 | 67.4 | 32.7 | 48.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 72.6 | — | — | — | — | — | — | |
| CriticalKV (SnapKV)Model=Mistral-7B, Cache Budget=40%, Base Method=SnapKV2025.02 | 45.35 | 34.33 | 36.61 | 26.94 | 68.57 | 51.05 | 62.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.3 | — | |
| AdaKVModel=Mistral-7B, Cache Budget=40%2025.02 | 45.07 | 33.42 | 35.97 | 26.24 | 68.67 | 50.75 | 63.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 4.9 | — | |
| HeadKVModel=Mistral-7B, Cache Budget=40%2025.02 | 44.84 | 33.9 | 35.28 | 26.76 | 67.46 | 50.5 | 63.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 5.4 | — | |
| SNAPKVBackbone=Qwen3-4B, C.R.=0.52026.04 | 44.48 | — | — | — | — | — | — | — | — | — | 58.28 | 41.74 | 32.09 | 29.92 | 23.32 | 23.7 | 45.5 | 87.52 | 40.16 | 4.5 | — | 63.87 | 58.54 | — | 27.09 | 43.38 | 99 | 33.06 | — | — | |
| SnapKVModel=Mistral-7B, Cache Budget=40%2025.02 | 44.03 | 31.64 | 33.57 | 26.24 | 67.95 | 49.75 | 63.41 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.1 | — | |
| CriticalKV (HeadKV)Model=Llama-3.1-8B, Cache Budget=20%, Base Method=HeadKV2025.02 | 43.99 | 33.6 | 36.33 | 25.28 | 65.41 | 52.18 | 58.85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.6 | — | |
| CriticalKV (AdaKV)Model=Llama-3.1-8B, Cache Budget=20%, Base Method=AdaKV2025.02 | 43.77 | 32.74 | 35.31 | 24.98 | 64.74 | 52.3 | 61.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 11 | — | |
| CriticalKV (AdaKV)Model=Qwen2.5-32B, Cache Budget=20%, Base Method=AdaKV2025.02 | 43.75 | 29 | 47.4 | 24.06 | 67.44 | 52.75 | 45.43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.9 | — | |
| CAP(OURS)Backbone=Mistral0.3–7B, C.R. (Cache Ratio)=0.52026.04 | 43.64 | — | — | — | — | — | — | — | 37.38 | — | 47.59 | 36.4 | 27.2 | 33.63 | 24.99 | 26.7 | 54.5 | 87.12 | 38.32 | 5.28 | — | 52.42 | 57.85 | — | 26.64 | 49.18 | 93 | — | — | — | |
| CAP(OURS)Backbone=Mistral0.3–7B, C.R. (Cache Ratio)=0.252026.04 | 43.46 | — | — | — | — | — | — | — | 38.72 | — | 48.4 | 37.68 | 26.46 | 34.49 | 25.74 | 26.74 | 52.25 | 85.28 | 30.4 | 5.85 | — | 51.29 | 57.09 | — | 28.4 | 50.11 | 96.5 | — | — | — | |
| CriticalKV (HeadKV)Model=Mistral-7B, Cache Budget=20%, Base Method=HeadKV2025.02 | 43.46 | 30.95 | 34.47 | 25.63 | 67.58 | 48.02 | 61.76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 8.3 | — | |
| Qwen3 w/ MATCHSparsification Strategy=Post-sparsified LLM, SWA ratio=0.5, k=42026.06 | 43.4 | 44.8 | 39.1 | 20.7 | 61.7 | 50.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3 w/ MATCHSparsification Strategy=Post-sparsified LLM, SWA ratio=0.5, k=82026.06 | 43.4 | 44.7 | 38 | 20.6 | 62 | 51.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CriticalKV (SnapKV)Model=Qwen2.5-32B, Cache Budget=20%, Base Method=SnapKV2025.02 | 43.36 | 28.12 | 46.88 | 24.23 | 66.86 | 53 | 44.72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 10.8 | — | |
| KEYDIFFBackbone=Mistral0.3–7B, C.R. (Cache Ratio)=0.252026.04 | 43.3 | — | — | — | — | — | — | — | 38.74 | — | 46.12 | 39.57 | 26.95 | 33.67 | 25.61 | 26.64 | 55.05 | 86.4 | 25.64 | 3.73 | — | 50.14 | 56.89 | — | 27.12 | 52.01 | 98.5 | — | — | — | |
| Full AttnType=-, Comp.=1.00, Model=Llama-3.1-8B-Instruct2025.03 | 43.2 | 44.23 | 44.72 | 28.52 | 25.88 | 53.44 | 62.41 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Full AttnModel=Llama-3.1-8B-Instruct, Comp. (Total memory reduction)=1.002025.03 | 43.2 | 44.23 | 44.72 | 28.52 | 25.88 | 53.44 | 62.41 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MISTRAL0.3–7BBackbone=Mistral0.3–7B2026.04 | 43.18 | — | — | — | — | — | — | — | 40.2 | — | 48.95 | 36.85 | 27.74 | 34.53 | 25.38 | 26.89 | 55.75 | 85.09 | 20.52 | 5.46 | — | 49.72 | 56 | — | 28.29 | 51.49 | 98 | — | — | — | |
| HeadKVModel=Qwen2.5-32B, Cache Budget=20%2025.02 | 43.11 | 29.9 | 46.35 | 23.88 | 65.96 | 50.88 | 44.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 11.3 | — | |
| CriticalKV (SnapKV)Model=Llama-3.1-8B, Cache Budget=20%, Base Method=SnapKV2025.02 | 42.99 | 30.43 | 35.87 | 24.64 | 63.04 | 52.16 | 60.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 12.6 | — | |
| AdaKVModel=Llama-3.1-8B, Cache Budget=20%2025.02 | 42.87 | 31.39 | 34.9 | 24.29 | 63.7 | 50.39 | 61.14 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 12.9 | — | |
| EABackbone=Mistral0.3–7B, C.R. (Cache Ratio)=0.252026.04 | 42.85 | — | — | — | — | — | — | — | 39.23 | — | 49.12 | 36.57 | 26.35 | 33.97 | 25.03 | 26.87 | 55.75 | 86.42 | 21.22 | 4.09 | — | 51.34 | 55.3 | — | 26.9 | 50.98 | 96.5 | — | — | — | |
| CriticalKV (AdaKV)Model=Mistral-7B, Cache Budget=20%, Base Method=AdaKV2025.02 | 42.85 | 29.06 | 33.04 | 24.83 | 67.08 | 49.25 | 62.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 9.6 | — | |
| SnapKVType=Intra, Comp.=8.00, Model=Llama-3.1-8B-Instruct2025.03 | 42.83 | 43.17 | 44.13 | 26.09 | 27.75 | 53.27 | 62.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| xKV-SRModel=Llama-3.1-8B-Instruct, Comp. (Total memory reduction)=1.63, Effective GPU memory reduction=8.902025.03 | 42.69 | 44.38 | 44.63 | 27.98 | 25.55 | 52.13 | 61.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HeadKVModel=Llama-3.1-8B, Cache Budget=20%2025.02 | 42.64 | 31.53 | 34.97 | 24.49 | 62.79 | 49.32 | 61.14 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 13.3 | — | |
| QuestModel=Llama-3.1-8B-Instruct, Comp. (Total memory reduction)=1.00, Effective GPU memory reduction=8.002025.03 | 42.63 | 43.18 | 44.4 | 28.2 | 26.57 | 52.88 | 60.55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| KEYDIFFBackbone=Mistral0.3–7B, C.R. (Cache Ratio)=0.52026.04 | 42.53 | — | — | — | — | — | — | — | 34.49 | — | 45.05 | 36.52 | 25.03 | 32.22 | 24.69 | 25.79 | 52 | 88.11 | 35.84 | 3.85 | — | 46.75 | 59.53 | — | 26.09 | 49.54 | 95 | — | — | — | |
| xK-SRModel=Llama-3.1-8B-Instruct, Comp. (Total memory reduction)=1.68, Effective GPU memory reduction=10.452025.03 | 42.5 | 43.64 | 44.47 | 27.62 | 25.31 | 52.63 | 61.32 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PyramidKVType=Intra, Comp.=8.00, Model=Llama-3.1-8B-Instruct2025.03 | 42.47 | 42.92 | 43.99 | 25.73 | 27.62 | 53.02 | 61.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| xKV-SRModel=Llama-3.1-8B-Instruct, Comp. (Total memory reduction)=5.352025.03 | 42.4 | 44.58 | 45.2 | 27.76 | 25.32 | 52.63 | 58.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SNAPKVBackbone=Mistral0.3–7B, C.R. (Cache Ratio)=0.252026.04 | 42.35 | — | — | — | — | — | — | — | 36.74 | — | 47.43 | 37.68 | 26.35 | 33.45 | 24.58 | 26.2 | 53.75 | 85.09 | 20.79 | 5.66 | — | 50.99 | 56.59 | — | 26.13 | 48.13 | 98 | — | — | — | |
| xKVType=Inter, Comp.=8.03, Model=Llama-3.1-8B-Instruct2025.03 | 42.27 | 44.39 | 38.82 | 26.14 | 27.34 | 55.5 | 61.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |