# $\\textit{Agents Under Siege}$: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks

Source: [$\\textit{Agents Under Siege}$: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks](<https://www.sota2.com/research/paper/68fa7c001e209131cea8bd0d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T10:59:25.294Z

Authors: Rana Muhammad Shahroz Khan, Zhen Tan, Sukwon Yun, Charles Fleming, Tianlong Chen

Published: 2025-03-31T20:43:56.000Z

## Abstract

Most discussions about Large Language Model (LLM) safety have focused on single-agent settings but multi-agent LLM systems now create novel adversarial risks because their behavior depends on communication between agents and decentralized reasoning. In this work, we innovatively focus on attacking pragmatic systems that have constrains such as limited token bandwidth, latency between message delivery, and defense mechanisms. We design a $\\textit{permutation-invariant adversarial attack}$ that optimizes prompt distribution across latency and bandwidth-constraint network topologies to bypass distributed safety mechanisms within the system. Formulating the attack path as a problem of $\\textit{maximum-flow minimum-cost}$, coupled with the novel $\\textit{Permutation-Invariant Evasion Loss (PIEL)}$, we leverage graph-based optimization to maximize attack success rate while minimizing detection risk. Evaluating across models including $\\texttt{Llama}$, $\\texttt{Mistral}$, $\\texttt{Gemma}$, $\\texttt{DeepSeek}$ and other variants on various datasets like $\\texttt{JailBreakBench}$ and $\\texttt{AdversarialBench}$, our method outperforms conventional attacks by up to $7\\times$, exposing critical vulnerabilities in multi-agent systems. Moreover, we demonstrate that existing defenses, including variants of $\\texttt{Llama-Guard}$ and $\\texttt{PromptGuard}$, fail to prohibit our attack, emphasizing the urgent need for multi-agent specific safety mechanisms.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Jailbreak Attack on JailbreakBenchmark](<https://www.sota2.com/research/sota/jailbreak-attack-on-jailbreakbenchmark.md>): Jailbreak Attack; JailbreakBenchmark; ASR (m): 0.78.

- [Jailbreak Attack on AdversarialBenchmark](<https://www.sota2.com/research/sota/jailbreak-attack-on-adversarialbenchmark.md>): Jailbreak Attack; AdversarialBenchmark; ASR (m): 0.512.

- [Jailbreak Attack on In-the-wild Jailbreak](<https://www.sota2.com/research/sota/jailbreak-attack-on-in-the-wild-jailbreak.md>): Jailbreak Attack; In-the-wild Jailbreak; ASR (m): 60.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2504.00218>)

[PDF](<https://arxiv.org/pdf/2504.00218.pdf>)