# Memory-based Deep Reinforcement Learning for POMDPs

Source: [Memory-based Deep Reinforcement Learning for POMDPs](<https://www.sota2.com/research/paper/68fa6acf1e209131cea2a6f9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:33:16.767Z

Authors: Lingheng Meng, Rob Gorbet, Dana Kuli\\'c

Published: 2021-02-24T15:25:13.000Z

## Abstract

A promising characteristic of Deep Reinforcement Learning (DRL) is its capability to learn optimal policy in an end-to-end manner without relying on feature engineering. However, most approaches assume a fully observable state space, i.e. fully observable Markov Decision Processes (MDPs). In real-world robotics, this assumption is unpractical, because of issues such as sensor sensitivity limitations and sensor noise, and the lack of knowledge about whether the observation design is complete or not. These scenarios lead to Partially Observable MDPs (POMDPs). In this paper, we propose Long-Short-Term-Memory-based Twin Delayed Deep Deterministic Policy Gradient (LSTM-TD3) by introducing a memory component to TD3, and compare its performance with other DRL algorithms in both MDPs and POMDPs. Our results demonstrate the significant advantages of the memory component in addressing POMDPs, including the ability to handle missing and noisy observation data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Robotic Control on Ant P-FLK](<https://www.sota2.com/research/sota/robotic-control-on-ant-p-flk.md>): Robotic Control; Ant P-FLK; Maximum Average Return: 2658.81.

- [Robotic Control on Ant MDP](<https://www.sota2.com/research/sota/robotic-control-on-ant-mdp.md>): Robotic Control; Ant MDP; Max Average Return: 4070.03.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2102.12344>)

[PDF](<https://arxiv.org/pdf/2102.12344.pdf>)
