Long-context language understanding on LongBench (Comprehensive 16-Task Subset)
30.46NtrvQAFull Cache
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Full CacheBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%2026.05 | 30.46 | 47.29 | 55.03 | 58.65 | 51.61 | 33.03 | 35.03 | 24.97 | 27.04 | 29 | 84.81 | 39.29 | 10.15 | 100 | 51.43 | 44.19 | 45.12 | |
| CaMBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 29.29 | 30.05 | 39.26 | 53.79 | 39 | 27.6 | 28.51 | 22.76 | 23.42 | 22 | 82.55 | 39.45 | 8.79 | 96.5 | 46.94 | 44.19 | 39.63 | |
| GRKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 29.24 | 32.87 | 39.86 | 53.47 | 42.68 | 26.13 | 29.27 | 23.04 | 23.56 | 50 | 81.83 | 41.68 | 10.05 | 97 | 53.55 | 46.15 | 42.52 | |
| CriticalKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%2026.05 | 28.94 | 31.35 | 39.22 | 53.57 | 41.79 | 25.94 | 29.27 | 23.03 | 23.4 | 49 | 80.58 | 41.01 | 9.55 | 96.5 | 52.89 | 45.99 | 42 | |
| Full CacheBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%2026.05 | 28.38 | 40.72 | 51.68 | 49 | 36.21 | 27.81 | 34.23 | 25.41 | 26.93 | 55.75 | 84.93 | 20.94 | 5.05 | 98 | 46.92 | 53.86 | 42.86 | |
| SnapKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%2026.05 | 27.84 | 27.19 | 34.18 | 54.98 | 37.9 | 24.48 | 27.76 | 21.86 | 22.72 | 38 | 83.01 | 40.94 | 7.55 | 89.5 | 51.73 | 44.31 | 39.62 | |
| GRKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 27.73 | 28.3 | 34.36 | 54.45 | 36.85 | 25.99 | 27.98 | 22.04 | 23.11 | 41.5 | 84.06 | 41.17 | 8.55 | 92 | 51.47 | 44.86 | 40.28 | |
| D2OBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 27.18 | 26.55 | 31.28 | 49.36 | 31.41 | 22.61 | 25.97 | 21.18 | 21.95 | 36 | 82.81 | 39.93 | 10 | 74.5 | 50.85 | 45.46 | 37.32 | |
| CaMBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 26.16 | 26.61 | 35.54 | 54.42 | 37.46 | 26 | 27.03 | 21.87 | 22.59 | 20.07 | 84.78 | 39.56 | 8.68 | 86.5 | 46.53 | 42.72 | 37.91 | |
| AsymKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 25.6 | 30.06 | 36.48 | 49.08 | 33.52 | 19.46 | 27.92 | 22.29 | 24.4 | 17.5 | 89.24 | 26.99 | 5.61 | 63.5 | 49.1 | 44.95 | 35.36 | |
| D2OBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 25.11 | 19.71 | 29.16 | 48.41 | 25.52 | 19.83 | 24.91 | 19.93 | 20.88 | 33 | 82.41 | 38.91 | 7.04 | 66.5 | 50.08 | 45.05 | 34.78 | |
| AsymKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 24.57 | 24.1 | 35.83 | 42.84 | 28.98 | 16.77 | 28.72 | 21.64 | 23.78 | 45.82 | 85.61 | 24.71 | 1.61 | 60.17 | 45.81 | 48.68 | 34.98 | |
| GRKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 23.2 | 25.84 | 43.02 | 44.07 | 32.92 | 19.64 | 29.07 | 23.08 | 23.69 | 47 | 88.03 | 24.3 | 4.88 | 93.5 | 50.49 | 52.02 | 39.05 | |
| AsymKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 23.06 | 24.27 | 37.25 | 45.63 | 26.43 | 18.94 | 28.8 | 21.58 | 24.07 | 51 | 85.36 | 23.25 | 2.43 | 54.5 | 45.06 | 49.23 | 35.05 | |
| CaMBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 22.93 | 24.64 | 41.2 | 45.48 | 33.7 | 22.26 | 28.27 | 22.86 | 23.42 | 35.07 | 85.44 | 11.63 | 4.25 | 90 | 48.95 | 50.17 | 36.89 | |
| CriticalKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%2026.05 | 22.88 | 25.68 | 42.55 | 42.53 | — | 18.68 | 28.76 | 23.02 | 23.22 | 47.25 | 86.99 | 23.14 | 4.65 | 93 | 49.49 | 51.65 | 38.47 | |
| AsymKVBackbone=Llama-3.1-8B-Instruct, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 22.76 | 29.76 | 34.17 | 43.08 | 34.27 | 16.6 | 27.21 | 22.12 | 24.24 | 7 | 83.87 | 21.86 | 3.83 | 45.5 | 51.92 | 47.17 | 32.21 | |
| CaMBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 19.5 | 18.3 | 35.13 | 40.35 | 21.94 | 20.41 | 27.57 | 21.88 | 22.81 | 36.5 | 84.68 | 14.04 | 2.47 | 87.17 | 47.69 | 50.85 | 34.46 | |
| D2OBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=CriticalKV2026.05 | 19.28 | 15.96 | 35.13 | 36.64 | 25.84 | 16.48 | 26.28 | 21.13 | 21.76 | 41.5 | 87.08 | 26.98 | 4.43 | 57.5 | 50.61 | 51.26 | 33.62 | |
| GRKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 18.7 | 19.45 | 37.81 | 39.5 | 27.82 | 21.44 | 28.03 | 22.15 | 23 | 44 | 87.17 | 23.13 | 4.65 | 90 | 52.51 | 52.54 | 36.99 | |
| SnapKVBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%2026.05 | 18.06 | 18.46 | 36.68 | 39.7 | 25.67 | 20.95 | 27.83 | 21.68 | 22.62 | 43.75 | 86.42 | 23.42 | 4.55 | 89.5 | 51.47 | 52.6 | 36.46 | |
| D2OBackbone=Mistral-7B-Instruct-v0.3, Cache Budget=20%, Base Eviction Method=SnapKV2026.05 | 16.28 | 10.77 | 34.09 | 35.27 | 22.61 | 14.74 | 25.59 | 19.93 | 20.92 | 35.75 | 86.47 | 28.09 | 5.26 | 55 | 50.65 | 52.16 | 32.1 |