# Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning

Source: [Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning](<https://www.sota2.com/research/paper/68fa6a1e1e209131cea260c4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:16:37.291Z

Authors: Luisa Zintgraf, Leo Feng, Cong Lu, Maximilian Igl, Kristian Hartikainen, Katja Hofmann, Shimon Whiteson

Published: 2020-10-02T15:43:31.000Z

## Abstract

To rapidly learn a new task, it is often essential for agents to explore efficiently -- especially when performance matters from the first timestep. One way to learn such behaviour is via meta-learning. Many existing methods however rely on dense rewards for meta-training, and can fail catastrophically if the rewards are sparse. Without a suitable reward signal, the need for exploration during meta-training is exacerbated. To address this, we propose HyperX, which uses novel reward bonuses for meta-training to explore in approximate hyper-state space (where hyper-states represent the environment state and the agent's task belief). We show empirically that HyperX meta-learns better task-exploration and adapts more successfully to new tasks than existing methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Cumulative Reward Maximization on Ray Maze](<https://www.sota2.com/research/sota/cumulative-reward-maximization-on-ray-maze.md>): Cumulative Reward Maximization; Ray Maze; Mean Cumulative Reward: 0.07.

- [Cumulative Reward Maximization on Dark Treasure Rooms](<https://www.sota2.com/research/sota/cumulative-reward-maximization-on-dark-treasure-rooms.md>): Cumulative Reward Maximization; Dark Treasure Rooms; Mean Cumulative Reward: -0.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2010.01062>)

[PDF](<https://arxiv.org/pdf/2010.01062.pdf>)
