# The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPs Source: [The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPs]() Collection index: [llms.txt]() Data updated: 2025-10-25T15:45:26.114Z Authors: Johannes M\\"uller, Guido Mont\\'ufar Published: 2021-10-14T14:42:09.000Z ## Abstract We consider the problem of finding the best memoryless stochastic policy for an infinite-horizon partially observable Markov decision process (POMDP) with finite state and action spaces with respect to either the discounted or mean reward criterion. We show that the (discounted) state-action frequencies and the expected cumulative reward are rational functions of the policy, whereby the degree is determined by the degree of partial observability. We then describe the optimization problem as a linear optimization problem in the space of feasible state-action frequencies subject to polynomial constraints that we characterize explicitly. This allows us to address the combinatorial and geometric complexity of the optimization problem using recent tools from polynomial optimization. In particular, we estimate the number of critical points and use the polynomial programming description of reward maximization to solve a navigation problem in a grid world. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [POMDP Policy Optimization on Random POMDP](): POMDP Policy Optimization; Random POMDP; L^pi: 3.2125. ## Original sources and implementation [Original paper]() [PDF]()