# Memoryless Policy Iteration for Episodic POMDPs Source: [Memoryless Policy Iteration for Episodic POMDPs]() Collection index: [llms.txt]() Data updated: 2025-12-16T05:02:15.912Z Authors: Roy van Zuijlen, Duarte Antunes Published: 2025-12-11T19:54:57.000Z ## Abstract Memoryless and finite-memory policies offer a practical alternative for solving partially observable Markov decision processes (POMDPs), as they operate directly in the output space rather than in the high-dimensional belief space. However, extending classical methods such as policy iteration to this setting remains difficult; the output process is non-Markovian, making policy-improvement steps interdependent across stages. We introduce a new family of monotonically improving policy-iteration algorithms that alternate between single-stage output-based policy improvements and policy evaluations according to a prescribed periodic pattern. We show that this family admits optimal patterns that maximize a natural computational-efficiency index, and we identify the simplest pattern with minimal period. Building on this structure, we further develop a model-free variant that estimates values from data and learns memoryless policies directly. Across several POMDPs examples, our method achieves significant computational speedups over policy-gradient baselines and recent specialized algorithms in both model-based and model-free settings. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [POMDP Policy Optimization on Random POMDP](): POMDP Policy Optimization; Random POMDP; L^pi: 3.2125. ## Original sources and implementation [Original paper]() [PDF]()