# Observable Channels, Not Just Storage: Evaluating Privacy Leakage in LLM Agent Pipelines

Source: [Observable Channels, Not Just Storage: Evaluating Privacy Leakage in LLM Agent Pipelines](<https://www.sota2.com/research/paper/69c36c040636e5bf4afc64a4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-01T05:00:12.613Z

Authors: Tao Huang, Chen Hou, Guosen Wu, Jiayang Meng

Published: 2026-03-24T03:29:39.000Z

## Abstract

Privacy leakage in LLM agents is often studied through individual storage or execution components, such as memory modules, retrieval pipelines, or tool-mediated artifacts. However, these settings are typically analyzed in isolation, making it difficult to compare how private internal dependence becomes externally recoverable across heterogeneous agent pipelines. In this paper, we present CIPL (Channel Inversion for Privacy Leakage) as a unified channel-oriented measurement interface for evaluating privacy leakage in LLM agent pipelines. Rather than claiming a universally strongest attack recipe, CIPL provides a shared way to represent a target through its sensitive source, selection, assembly, execution, observation, and extraction stages, and to measure how internal exposure is transformed into attacker-recoverable leakage under a common protocol. Using memory-based, retrieval-mediated, and tool-mediated instantiations under this shared interface, we identify a distinct cross-target risk picture. Memory behaves as a near-saturated high-risk special case, while beyond-memory leakage exhibits a different regime: retrieval-mediated targets show frequent but often incomplete leakage, and tool-mediated targets are strongly shaped by the exposed observation surface and provider behavior. We further show that leakage is governed by channel conditions rather than by a universally dominant recipe: cleaned weak controls sharply suppress leakage, and semantic annotation reveals attacker-useful leakage beyond exact-match extraction. Together, these findings suggest that privacy risk in LLM agent pipelines is better understood through \\emph{observable channels}, not just storage components. More broadly, our results motivate channel-oriented privacy evaluation as a necessary complement to component-local or exact-only analyses.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Cross-Input Privacy Leakage on memory\_ehr](<https://www.sota2.com/research/sota/cross-input-privacy-leakage-on-memory_ehr.md>): Cross-Input Privacy Leakage; memory\_ehr; RN: not recorded.

- [Cross-Input Privacy Leakage on memory\_rap](<https://www.sota2.com/research/sota/cross-input-privacy-leakage-on-memory_rap.md>): Cross-Input Privacy Leakage; memory\_rap; RN: not recorded.

- [Cross-Input Privacy Leakage on rag toy](<https://www.sota2.com/research/sota/cross-input-privacy-leakage-on-rag-toy.md>): Cross-Input Privacy Leakage; rag toy; RN Count: not recorded.

- [Cross-Input Privacy Leakage on tool\_mock (args\_exfil, llm)](<https://www.sota2.com/research/sota/cross-input-privacy-leakage-on-tool_mock-args_exfil-llm.md>): Cross-Input Privacy Leakage; tool\_mock (args\_exfil, llm); RN: not recorded.

- [Cross-Input Privacy Leakage on tool\_mock return\_echo, llm](<https://www.sota2.com/research/sota/cross-input-privacy-leakage-on-tool_mock-return_echo-llm.md>): Cross-Input Privacy Leakage; tool\_mock return\_echo, llm; RN: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2603.22751>)

[PDF](<https://arxiv.org/pdf/2603.22751.pdf>)
