# Risk-seeking conservative policy iteration with agent-state based policies for Dec-POMDPs with guaranteed convergence Source: [Risk-seeking conservative policy iteration with agent-state based policies for Dec-POMDPs with guaranteed convergence]() Collection index: [llms.txt]() Data updated: 2026-04-14T05:00:11.147Z Authors: Amit Sinha, Matthieu Geist, Aditya Mahajan Published: 2026-04-10T17:05:47.000Z ## Abstract Optimally solving decentralized decision-making problems modeled as Dec-POMDPs is known to be NEXP-complete. These optimal solutions are policies based on the entire history of observations and actions of an agent. However, some applications may require more compact policies because of limited compute capabilities, which can be modeled by considering a limited number of memory states (or agent states). While such an agent-state based policy class may not contain the optimal solution, it is still of practical interest to find the best agent-state policy within the class. We focus on an iterated best response style algorithm which guarantees monotonic improvements and convergence to a local optimum in polynomial runtime in the Dec-POMDP model size. In order to obtain a better local optimum, we use a modified objective which incentivizes risk-seeking alongside a conservative policy iteration update. Our empirical results show that our approach performs as well as state-of-the-art approaches on several benchmark Dec-POMDPs, achieving near-optimal performance while having polynomial runtime despite the limited memory. We also show that using more agent states (a larger memory) leads to greater performance. Our approach provides a novel way of incorporating memory constraints on the agents in the Dec-POMDP problem. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Dec-POMDP Planning on MARS ROVERS](): Dec-POMDP Planning; MARS ROVERS; Cumulative Reward: 252.63. - [Dec-POMDP Planning on COOPERATIVE BOX PUSHING](): Dec-POMDP Planning; COOPERATIVE BOX PUSHING; Cumulative Reward: 2352.18. - [Dec-POMDP Planning on RECYCLING ROBOTS](): Dec-POMDP Planning; RECYCLING ROBOTS; Cumulative Reward: 6154.56. - [Dec-POMDP Planning on DEC TIGER](): Dec-POMDP Planning; DEC TIGER; Cumulative Reward: 120.3. ## Original sources and implementation [Original paper]() [PDF]()