# G-KV: Decoding-Time KV Cache Eviction with Global Attention Source: [G-KV: Decoding-Time KV Cache Eviction with Global Attention]() Collection index: [llms.txt]() Data updated: 2025-12-03T05:02:38.198Z Authors: Mengqi Liao, Lu Wang, Chaoyun Zhang, Zekai Shen, Xiaowei Mao, Si Qin, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Huaiyu Wan Published: 2025-11-29T14:21:33.000Z ## Abstract Recent reasoning large language models (LLMs) excel in complex tasks but encounter significant computational and memory challenges due to long sequence lengths. KV cache compression has emerged as an effective approach to greatly enhance the efficiency of reasoning. However, existing methods often focus on prompt compression or token eviction with local attention score, overlooking the long-term importance of tokens. We propose G-KV, a KV cache eviction method that employs a global scoring mechanism, combining local and historical attention scores to more accurately assess token importance. Additionally, we introduce post-training techniques, including reinforcement learning and distillation, to optimize models for compressed KV cache settings. The code of this paper is available on: https://github.com/microsoft/G-KV. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Inference Efficiency on DeepScaleR-40k (1,024 mathematical problems)](): Inference Efficiency; DeepScaleR-40k (1,024 mathematical problems); Throughput (tokens/s): 760.74. ## Original sources and implementation [Original paper]() [PDF]()