# Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilities

Source: [Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilities](<https://www.sota2.com/research/paper/68fa5a6f1e209131ce9bfdd0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:32:52.592Z

Authors: Donghao Ying, Yunkai Zhang, Yuhao Ding, Alec Koppel, Javad Lavaei

Published: 2023-05-27T20:08:35.000Z

## Abstract

We investigate safe multi-agent reinforcement learning, where agents seek to collectively maximize an aggregate sum of local objectives while satisfying their own safety constraints. The objective and constraints are described by {\\it general utilities}, i.e., nonlinear functions of the long-term state-action occupancy measure, which encompass broader decision-making goals such as risk, exploration, or imitations. The exponential growth of the state-action space size with the number of agents presents challenges for global observability, further exacerbated by the global coupling arising from agents' safety constraints. To tackle this issue, we propose a primal-dual method utilizing shadow reward and $\\kappa$-hop neighbor truncation under a form of correlation decay property, where $\\kappa$ is the communication radius. In the exact setting, our algorithm converges to a first-order stationary point (FOSP) at the rate of $\\mathcal{O}\\left(T^{-2/3}\\right)$. In the sample-based setting, we demonstrate that, with high probability, our algorithm requires $\\widetilde{\\mathcal{O}}\\left(\\epsilon^{-3.5}\\right)$ samples to achieve an $\\epsilon$-FOSP with an approximation error of $\\mathcal{O}(\\phi\_0^{2\\kappa})$, where $\\phi\_0\\in (0,1)$. Finally, we demonstrate the effectiveness of our model through extensive numerical experiments.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Safe Multi-Agent Reinforcement Learning on Pistonball 10 agents](<https://www.sota2.com/research/sota/safe-multi-agent-reinforcement-learning-on-pistonball-10-agents.md>): Safe Multi-Agent Reinforcement Learning; Pistonball 10 agents; Constraint Violation Rate: 4.919.

- [Safe Multi-Agent Reinforcement Learning on Wireless Communication 25 agents](<https://www.sota2.com/research/sota/safe-multi-agent-reinforcement-learning-on-wireless-communication-25-agents.md>): Safe Multi-Agent Reinforcement Learning; Wireless Communication 25 agents; Constraint Violation Rate: 19.26.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2305.17568>)

[PDF](<https://arxiv.org/pdf/2305.17568.pdf>)