# VGGT-Segmentor: Geometry-Enhanced Cross-View Segmentation

Source: [VGGT-Segmentor: Geometry-Enhanced Cross-View Segmentation](<https://www.sota2.com/research/paper/69e06d277d0172140ec7cbaf>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-26T05:00:05.667Z

Authors: Yulu Gao, Bohao Zhang, Zongheng Tang, Jitong Liao, Wenjun Wu, Si Liu

Published: 2026-04-15T08:05:13.000Z

## Abstract

Instance-level object segmentation across disparate egocentric and exocentric views is a fundamental challenge in visual understanding, critical for applications in embodied AI and remote collaboration. This task is exceptionally difficult due to severe changes in scale, perspective, and occlusion, which destabilize direct pixel-level matching. While recent geometry-aware models like VGGT provide a strong foundation for feature alignment, we find they often fail at dense prediction tasks due to significant pixel-level projection drift, even when their internal object-level attention remains consistent. To bridge this gap, we introduce VGGT-Segmentor (VGGT-S), a framework that unifies robust geometric modeling with pixel-accurate semantic segmentation. VGGT-S leverages VGGT's powerful cross-view feature representation and introduces a novel Union Segmentation Head. This head operates in three stages: mask prompt fusion, point-guided prediction, and iterative mask refinement, effectively translating high-level feature alignment into a precise segmentation mask. Furthermore, we propose a single-image self-supervised training strategy that eliminates the need for paired annotations and enables strong generalization. On the Ego-Exo4D benchmark, VGGT-S sets a new state-of-the-art, achieving 67.7% and 68.0% average IoU for Ego to Exo and Exo to Ego tasks, respectively, significantly outperforming prior methods. Notably, our correspondence-free pretrained model surpasses most fully-supervised baselines, demonstrating the effectiveness and scalability of our approach. Code is publicly available at: https://github.com/buaa-colalab/VGGT-S.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Cross-view Instance Segmentation on Ego-Exo4D Ego-to-Exo](<https://www.sota2.com/research/sota/cross-view-instance-segmentation-on-ego-exo4d-ego-to-exo.md>): Cross-view Instance Segmentation; Ego-Exo4D Ego-to-Exo; IoU: 67.7.

- [Cross-view Instance Segmentation on Ego-Exo4D Exo-to-Ego](<https://www.sota2.com/research/sota/cross-view-instance-segmentation-on-ego-exo4d-exo-to-ego.md>): Cross-view Instance Segmentation; Ego-Exo4D Exo-to-Ego; IoU: 68.

- [Cross-view Instance Segmentation on MvMHAT](<https://www.sota2.com/research/sota/cross-view-instance-segmentation-on-mvmhat.md>): Cross-view Instance Segmentation; MvMHAT; AP: 80.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.13596>)

[PDF](<https://arxiv.org/pdf/2604.13596.pdf>)
