# Seek to Segment: Active Perception for Panoramic Referring Segmentation

Source: [Seek to Segment: Active Perception for Panoramic Referring Segmentation](<https://www.sota2.com/research/paper/6a47421a143ebdcacacc5e3a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-04T05:12:22.129Z

Authors: Song Tang, Shuming Hu, Xincheng Shuai, Henghui Ding, Yu-Gang Jiang

Published: 2026-07-02T17:56:49.000Z

## Abstract

Existing referring segmentation models passively process static images captured from fixed perspectives, limiting their applicability in Embodied AI, where agents must perform active perception in the continuous 360$^\\circ$ environments. To bridge this gap, we introduce a novel task: Active Panoramic Referring Segmentation (APRS). In this setting, an agent is required to adjust its viewing direction ($\\Delta\\theta, \\Delta\\phi$) to explore the 360$^\\circ$ environment, seeking the object specified by a user instruction for segmentation. To tackle this challenging task, we propose PanoSeeker, a memory-augmented agent for efficient APRS. Rather than relying on heuristic scanning, PanoSeeker integrates a Vision-Language Model (VLM) with EgoSphere, an explicit spatial visual memory. By progressively integrating sequential local observations into a unified 360$^\\circ$ representation, EgoSphere enables the agent to plan efficient and non-redundant search trajectories. Once the target is found, the agent performs active viewpoint alignment and outputs the segmentation mask. Furthermore, we curate an expert-annotated search trajectory dataset with memory timelines for Supervised Fine-Tuning, followed by Reinforcement Learning post-training to explicitly optimize PanoSeeker's exploration efficiency. Extensive experiments on our newly established APRS benchmark demonstrate that PanoSeeker achieves superior search efficiency and segmentation accuracy, significantly outperforming adapted state-of-the-art baselines.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Active Panoramic Referring Expression Segmentation on APRS (test)](<https://www.sota2.com/research/sota/active-panoramic-referring-expression-segmentation-on-aprs-test.md>): Active Panoramic Referring Expression Segmentation; APRS (test); SR: 75.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2607.02497>)

[PDF](<https://arxiv.org/pdf/2607.02497.pdf>)
