# Beyond Monotonicity: Revisiting Factorization Principles in Multi-Agent Q-Learning Source: [Beyond Monotonicity: Revisiting Factorization Principles in Multi-Agent Q-Learning]() Collection index: [llms.txt]() Data updated: 2026-03-24T05:00:03.763Z Authors: Tianmeng Hu, Yongzheng Cui, Rui Tang, Biao Luo, Ke Li Published: 2025-11-12T22:49:35.000Z ## Abstract Value decomposition is a central approach in multi-agent reinforcement learning (MARL), enabling centralized training with decentralized execution by factorizing the global value function into local values. To ensure individual-global-max (IGM) consistency, existing methods either enforce monotonicity constraints, which limit expressive power, or adopt softer surrogates at the cost of algorithmic complexity. In this work, we present a dynamical systems analysis of non-monotonic value decomposition, modeling learning dynamics as continuous-time gradient flow. We prove that, under approximately greedy exploration, all zero-loss equilibria violating IGM consistency are unstable saddle points, while only IGM-consistent solutions are stable attractors of the learning dynamics. Extensive experiments on both synthetic matrix games and challenging MARL benchmarks demonstrate that unconstrained, non-monotonic factorization reliably recovers IGM-optimal solutions and consistently outperforms monotonic baselines. Additionally, we investigate the influence of temporal-difference targets and exploration strategies, providing actionable insights for the design of future value-based MARL algorithms. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Value Function Estimation on Game B Matrix Game](): Value function estimation; Game B Matrix Game; Estimated Payoff: 12. - [Value function estimation on Matrix Game B v1 (test)](): Value function estimation; Matrix Game B v1 (test); Q1 Value (A): 6.2. - [Matrix Game on Game A](): Matrix Game; Game A; Joint Payoff Estimate: 12. - [Matrix Game on Game B](): Matrix Game; Game B; Estimated Joint Payoff: 12. - [Matrix Game Value Estimation on Game A](): Matrix Game Value Estimation; Game A; Estimated Joint Value (Q\_tot): not recorded. - [Value function estimation on Game B matrix (train)](): Value function estimation; Game B matrix (train); Estimated Payoff: not recorded. ## Original sources and implementation [Original paper]() [PDF]()