Long-context language modeling on RULER Sequence length = 64k
100S-NIAH Score (Component 1)RDKV
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RDKVBackbone=LLaMA-3.1-8B-Instruct, KV cache budget (Btotal)=1024L2026.05 | 100 | 98.4 | 99 | 99 | 93 | 30.6 | 98.6 | 93 | 92.9 | 6.6 | 69.7 | 80.1 | — | — | |
| Full AttnBackbone=Llama-3.1-8B-Instruct, KV-Cache reduction (Comp.)=1.002025.03 | 100 | 100 | — | 98.96 | 97.92 | — | 98.96 | 97.66 | 97.29 | — | 85.42 | 91.89 | 83.33 | 59.38 | |
| ShadowKVBackbone=Llama-3.1-8B-Instruct, KV-Cache reduction (Comp.)=1.64, Effective GPU memory reduction=9.082025.03 | 100 | 100 | — | 98.96 | 97.92 | — | 96.88 | 94.53 | 66.04 | — | 74.65 | 87.17 | 82.29 | 60.42 | |
| xK-SR (Ours)Backbone=Llama-3.1-8B-Instruct, KV-Cache reduction (Comp.)=1.63, Effective GPU memory reduction=8.902025.03 | 100 | 100 | — | 98.96 | 97.92 | — | 98.44 | 95.31 | 87.92 | — | 74.65 | 89.7 | 83.33 | 60.42 | |
| ShadowKV‡Backbone=Llama-3.1-8B-Instruct, KV-Cache reduction (Comp.)=5.51, Variant=additional value cache compression2025.03 | 100 | 76.04 | — | 75 | 97.92 | — | 54.43 | 45.83 | 47.29 | — | 74.31 | 70.94 | 81.25 | 57.29 | |
| xKV-SR (Ours)Backbone=Llama-3.1-8B-Instruct, KV-Cache reduction (Comp.)=5.3522025.03 | 100 | 100 | — | 98.96 | 97.92 | — | 98.44 | 95.57 | 87.29 | — | 76.04 | 89.69 | 82.29 | 60.42 | |
| HqeKVBackbone=LLaMA-3.1-8B-Instruct, KV cache budget (Btotal)=1024L2026.05 | 99.6 | 96 | 72.4 | 97.2 | 94.8 | 28 | 86.4 | 81 | 86.2 | 0.1 | 84.1 | 75.1 | — | — | |
| QuestBackbone=Llama-3.1-8B-Instruct, KV-Cache reduction (Comp.)=1.00, Effective GPU memory reduction=8.002025.03 | 93.75 | 90.63 | — | 96.88 | 87.5 | — | 94.27 | 85.42 | 77.71 | — | 81.94 | 84.87 | 83.33 | 57.29 |