# RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation

Source: [RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation](<https://www.sota2.com/research/paper/6969c5e7e25ad2fa9d1ec75c>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-12T05:00:21.403Z

Authors: Yue Chang, Rufeng Chen, Zhaofan Zhang, Yi Chen, Yifan Tian, Sihong Xie

Published: 2026-01-15T08:15:01.000Z

## Abstract

Open-vocabulary 3D Scene Graph (3DSG) can enhance various downstream tasks in robotics by leveraging structured semantic representations, yet current 3DSG construction methods suffer from semantic inconsistencies caused by noisy cross-image aggregation under occlusions and constrained viewpoints. To mitigate the impact of such inconsistency, we propose RAG-3DSG, which introduces re-shot guided uncertainty estimation. By measuring the semantic consistency between original limited viewpoints and re-shot optimal viewpoints, this method quantifies the underlying semantic ambiguity of each graph object. Based on this quantification, we devise an Object-level Retrieval-Augmented Generation (RAG) that leverages low-uncertainty objects as semantic anchors to retrieve more reliable contextual knowledge, enabling a Vision-Language Model to rectify the predictions of uncertain objects and optimize the final 3DSG. Extensive evaluations across three challenging benchmarks and real-world robot trials demonstrate that RAG-3DSG achieves superior recall and precision, effectively mitigating semantic noise to provide highly reliable scene representations for robotics tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [3D Scene Graph Generation on Replica (room0-office4)](<https://www.sota2.com/research/sota/3d-scene-graph-generation-on-replica-room0-office4.md>): 3D scene graph generation; Replica (room0-office4); Node Precision: 88.

- [Semantic Segmentation on Replica 2019](<https://www.sota2.com/research/sota/semantic-segmentation-on-replica-2019.md>): Semantic segmentation; Replica 2019; mAcc: 40.67.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.10168>)

[PDF](<https://arxiv.org/pdf/2601.10168.pdf>)