Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

About

Learning when to communicate and doing that effectively is essential in multi-agent tasks. Recent works show that continuous communication allows efficient training with back-propagation in multi-agent scenarios, but have been restricted to fully-cooperative tasks. In this paper, we present Individualized Controlled Continuous Communication Model (IC3Net) which has better training efficiency than simple continuous communication model, and can be applied to semi-cooperative and competitive settings along with the cooperative settings. IC3Net controls continuous communication with a gating mechanism and uses individualized rewards foreach agent to gain better performance and scalability while fixing credit assignment issues. Using variety of tasks including StarCraft BroodWars explore and combat scenarios, we show that our network yields improved performance and convergence rates than the baselines as the scale increases. Our results convey that IC3Net agents learn when to communicate based on the scenario and profitability.

Amanpreet Singh, Tushar Jain, Sainbayar Sukhbaatar · 2018

Related benchmarks

Task	Dataset	Result
Multi-Agent Reinforcement Learning	SMAC terran_10v10 v2 (test)	AUC0.5	15
Pandemic Control	Hong Kong topology	Infection Status (I_n)25.9	7
Pandemic Control	New York topology	$I_n$ (Infection Rate)0.789	7
Cooperative Adaptive Cruise Control	CACC Catch-up scenario	RMSE (Lateral)8.687	7
Cooperative Adaptive Cruise Control	CACC Slow-down scenario	RMSE (Horizontal)7.877	7
Pandemic Control	Helsinki topology	Infection Rate (I_n)65.1	7
Multi-Agent Reinforcement Learning	Traffic Junction (TJ)	AUC55.8	6
Multi-Agent Reinforcement Learning	Traffic Junction TJ Hard	AUC46.4	6
Multi-Agent Reinforcement Learning	Hallway	AUC4.8	6
Multi-Agent Reinforcement Learning	Hallway group	AUC0	6