# Reciprocal Reward Influence Encourages Cooperation From Self-Interested Agents

Source: [Reciprocal Reward Influence Encourages Cooperation From Self-Interested Agents](<https://www.sota2.com/research/paper/68fa76981e209131cea6e081>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-02T08:37:52.096Z

Authors: John L. Zhou, Weizhe Hong, Jonathan C. Kao

Published: 2024-06-03T06:07:27.000Z

## Abstract

Cooperation between self-interested individuals is a widespread phenomenon in the natural world, but remains elusive in interactions between artificially intelligent agents. Instead, naive reinforcement learning algorithms typically converge to Pareto-dominated outcomes in even the simplest of social dilemmas. An emerging literature on opponent shaping has demonstrated the ability to reach prosocial outcomes by influencing the learning of other agents. However, such methods differentiate through the learning step of other agents or optimize for meta-game dynamics, which rely on privileged access to opponents' learning algorithms or exponential sample complexity, respectively. To provide a learning rule-agnostic and sample-efficient alternative, we introduce Reciprocators, reinforcement learning agents which are intrinsically motivated to reciprocate the influence of opponents' actions on their returns. This approach seeks to modify other agents' $Q$-values by increasing their return following beneficial actions (with respect to the Reciprocator) and decreasing it after detrimental actions, guiding them towards mutually beneficial actions without directly differentiating through a model of their policy. We show that Reciprocators can be used to promote cooperation in temporally extended social dilemmas during simultaneous learning. Our code is available at https://github.com/johnlyzhou/reciprocator/.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Iterated Prisoner's Dilemma on IPD-Analytic (round robin)](<https://www.sota2.com/research/sota/iterated-prisoner-s-dilemma-on-ipd-analytic-round-robin.md>): Iterated Prisoner's Dilemma; IPD-Analytic (round robin); Average Reward per Episode: -1.03.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.01641>)

[PDF](<https://arxiv.org/pdf/2406.01641.pdf>)
