# CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference

Source: [CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference](<https://www.sota2.com/research/paper/699e81ed6017c4460ffacda4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:00:04.574Z

Authors: Chao Fei, Guozhong Li, Chenxi Liu, Panos Kalnis

Published: 2026-02-24T09:54:59.000Z

## Abstract

Long-context LLMs demand accurate inference at low latency, yet decoding becomes primarily constrained by KV cache as context grows. Prior pruning methods are largely context-agnostic: their token selection ignores step-wise relevance and local semantics, which undermines quality. Moreover, their irregular accesses and selection overheads yield only limited wall-clock speedups. To address this, we propose \\textbf{CHESS}, an \\textit{algorithm-system co-design} KV-cache management system. Algorithmically, CHESS introduces a context-aware, hierarchical selection policy that dynamically reconstructs a coherent context for the current decoding. System-wise, coarse granularity selection eliminates expensive data movement, fully realizing practical acceleration from theoretical sparsity. Extensive evaluations demonstrate that CHESS surpasses Full-KV quality using only \\textbf{1\\%} of the KV cache, delivers low-latency stable inference with up to \\textbf{4.56$\\times$} higher throughput, and consistently outperforms other strong baselines. Code is available at \\href{https://anonymous.4open.science/r/CHESS-9958/}{https://anonymous.4open.science/r/CHESS/}.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Long-Context Inference on LongBench \<32K tokens v2 (short)](<https://www.sota2.com/research/sota/long-context-inference-on-longbench-32k-tokens-v2-short.md>): Long-Context Inference; LongBench \<32K tokens v2 (short); Accuracy: 41.7.

- [Long-Context Inference on LongBench 32K~128K tokens v2 (medium)](<https://www.sota2.com/research/sota/long-context-inference-on-longbench-32k-128k-tokens-v2-medium.md>): Long-Context Inference; LongBench 32K~128K tokens v2 (medium); Accuracy: 27.4.

- [Long-Context Inference on LongBench \>128K tokens v2 (long)](<https://www.sota2.com/research/sota/long-context-inference-on-longbench-128k-tokens-v2-long.md>): Long-Context Inference; LongBench \>128K tokens v2 (long); Accuracy: 33.3.

- [Long-Context Inference on LongBench v2 (hard)](<https://www.sota2.com/research/sota/long-context-inference-on-longbench-v2-hard.md>): Long-Context Inference; LongBench v2 (hard); Accuracy: 30.2.

- [Long Context Understanding on LongBench V2](<https://www.sota2.com/research/sota/long-context-understanding-on-longbench-v2.md>): Long-context Understanding; LongBench v2; Overall Score: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2602.20732>)

[PDF](<https://arxiv.org/pdf/2602.20732.pdf>)