# LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning

Source: [LongAttnComp: Cross-Family Context Compression for Long-Context Reasoning](<https://www.sota2.com/research/paper/6a1e640edbd7150506f85683>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-24T05:00:13.022Z

Authors: Mengmeng Ji, Ravi Shanker Raju, Jonathan Lingjie Li, Chen Wu

Published: 2026-05-31T16:40:36.000Z

## Abstract

As real-world applications increasingly require processing inputs of 100k+ tokens, the gap between context length and inference efficiency has become a critical bottleneck. Context compression offers a way to reduce prefill costs while preserving task accuracy. However, existing training-free attention-based methods leave substantial gaps in demanding long-context tasks such as code reasoning. We present LongAttnComp, a long-context adaptation of AttnComp that fine-tunes a lightweight cross-attention scoring layer and introduces tokenlevel chunking, a token-budget top-p algorithm, positional reordering, and a formatagnostic query parser. We further design a two-stage fine-tuning recipe for the compressor: Stage 1 builds a general retrieval foundation from NIAH-style data, and Stage 2 extends it with multi-hop and reasoning data for broader long-context task coverage. On InfiniteBench Code-Debug, LongAttnComp matches or exceeds full-context accuracy, substantially outperforms training-free baselines, and transfers across four target models from three families. On LongBench v2, the two-stage recipe largely closes the Stage 1 gap on multi-document reasoning while preserving Code-Debug performance.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Long-context code reasoning on InfiniteBench Code-Debug (test)](<https://www.sota2.com/research/sota/long-context-code-reasoning-on-infinitebench-code-debug-test.md>): Long-context code reasoning; InfiniteBench Code-Debug (test); Accuracy: 82.99.

- [Long-context Question Answering on LongBench v2 (full)](<https://www.sota2.com/research/sota/long-context-question-answering-on-longbench-v2-full.md>): Long-context Question Answering; LongBench v2 (full); Overall Accuracy: 49.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.01336>)

[PDF](<https://arxiv.org/pdf/2606.01336.pdf>)