# Learning Large-Scale Competitive Team Behaviors with Mean-Field Interactions and Online Opponent Modeling Source: [Learning Large-Scale Competitive Team Behaviors with Mean-Field Interactions and Online Opponent Modeling]() Collection index: [llms.txt]() Data updated: 2026-02-17T05:00:04.915Z Authors: Bhavini Jeloka, Yue Guan, Panagiotis Tsiotras Published: 2025-04-29T20:31:59.000Z ## Abstract While multi-agent reinforcement learning (MARL) has been proven effective across both collaborative and competitive tasks, existing algorithms often struggle to scale to large populations of agents. Recent advancements in mean-field (MF) theory provide scalable solutions by approximating population interactions as a continuum, yet most existing frameworks focus exclusively on either fully cooperative or purely competitive settings. To bridge this gap, we introduce MF-MAPPO, a mean-field extension of PPO designed for zero-sum team games that integrate intra-team cooperation with inter-team competition. MF-MAPPO employs a shared actor and a minimally informed critic per team and is trained directly on finite-population simulators, thereby enabling deployment to realistic scenarios with thousands of agents. We further show that MF-MAPPO naturally extends to partially observable settings through a simple gradient-regularized training scheme. Our evaluation utilizes large-scale benchmark scenarios using our own testing simulation platform for MF team games (MFEnv), including offense-defense battlefield tasks as well as variants of population-based rock-paper-scissors games that admit analytical solutions, for benchmarking. Across these benchmarks, MF-MAPPO outperforms existing methods and exhibits complex, heterogeneous behaviors, demonstrating the effectiveness of combining mean-field theory and MARL techniques at scale. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Mean Field Team Games competition on Battlefield 4x4 grid](): Mean Field Team Games competition; Battlefield 4x4 grid; Avg Reward: 86.26. - [Population-based Game Playing on Rock-Paper-Scissors (standard population-based)](): Population-based game playing; Rock-Paper-Scissors (RPS) standard population-based; Average Reward: 0. ## Original sources and implementation [Original paper]() [PDF]()