# HEADER: Hierarchical Robot Exploration via Attention-Based Deep Reinforcement Learning with Expert-Guided Reward

Source: [HEADER: Hierarchical Robot Exploration via Attention-Based Deep Reinforcement Learning with Expert-Guided Reward](<https://www.sota2.com/research/paper/68fa7c471e209131cea8d5e2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T10:38:22.670Z

Authors: Yuhong Cao, Yizhuo Wang, Jingsong Liang, Shuhao Liao, Yifeng Zhang, Peizhuo Li, Guillaume Sartoretti

Published: 2025-10-17T14:17:02.000Z

## Abstract

This work pushes the boundaries of learning-based methods in autonomous robot exploration in terms of environmental scale and exploration efficiency. We present HEADER, an attention-based reinforcement learning approach with hierarchical graphs for efficient exploration in large-scale environments. HEADER follows existing conventional methods to construct hierarchical representations for the robot belief/map, but further designs a novel community-based algorithm to construct and update a global graph, which remains fully incremental, shape-adaptive, and operates with linear complexity. Building upon attention-based networks, our planner finely reasons about the nearby belief within the local range while coarsely leveraging distant information at the global scale, enabling next-best-viewpoint decisions that consider multi-scale spatial dependencies. Beyond novel map representation, we introduce a parameter-free privileged reward that significantly improves model performance and produces near-optimal exploration behaviors, by avoiding training objective bias caused by handcrafted reward shaping. In simulated challenging, large-scale exploration scenarios, HEADER demonstrates better scalability than most existing learning and non-learning methods, while achieving a significant improvement in exploration efficiency (up to 20%) over state-of-the-art baselines. We also deploy HEADER on hardware and validate it in complex, large-scale real-life scenarios, including a 300m\*230m campus environment.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Autonomous Robotic Exploration on Warehouse Gazebo simulation (environment)](<https://www.sota2.com/research/sota/autonomous-robotic-exploration-on-warehouse-gazebo-simulation-environment.md>): Autonomous Robotic Exploration; Warehouse Gazebo simulation (environment); Exploration Distance (m): 492.

- [Robotic Exploration on Forest Gazebo simulation](<https://www.sota2.com/research/sota/robotic-exploration-on-forest-gazebo-simulation.md>): Robotic Exploration; Forest Gazebo simulation; Distance (m): 1230.

- [Robotic Exploration on Forest Gazebo simulation (test)](<https://www.sota2.com/research/sota/robotic-exploration-on-forest-gazebo-simulation-test.md>): Robotic Exploration; Forest Gazebo simulation (test); Distance (m): 123072.

- [Robotic Exploration on Warehouse Gazebo simulation (test)](<https://www.sota2.com/research/sota/robotic-exploration-on-warehouse-gazebo-simulation-test.md>): Robotic Exploration; Warehouse Gazebo simulation (test); Distance (m): 49217.

- [Robotic Exploration on Indoor Gazebo simulation (test)](<https://www.sota2.com/research/sota/robotic-exploration-on-indoor-gazebo-simulation-test.md>): Robotic Exploration; Indoor Gazebo simulation (test); Exploration Distance (m): 103040.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.15679>)

[PDF](<https://arxiv.org/pdf/2510.15679.pdf>)
