# Multiview Scene Graph

Source: [Multiview Scene Graph](<https://www.sota2.com/research/paper/68fa75bf1e209131cea69b27>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:56:51.478Z

Authors: Juexiao Zhang, Gao Zhu, Sihang Li, Xinhao Liu, Haorui Song, Xinran Tang, Chen Feng

Published: 2024-10-15T02:04:05.000Z

## Abstract

A proper scene representation is central to the pursuit of spatial intelligence where agents can robustly reconstruct and efficiently understand 3D scenes. A scene representation is either metric, such as landmark maps in 3D reconstruction, 3D bounding boxes in object detection, or voxel grids in occupancy prediction, or topological, such as pose graphs with loop closures in SLAM or visibility graphs in SfM. In this work, we propose to build Multiview Scene Graphs (MSG) from unposed images, representing a scene topologically with interconnected place and object nodes. The task of building MSG is challenging for existing representation learning methods since it needs to jointly address both visual place recognition, object detection, and object association from images with limited fields of view and potentially large viewpoint changes. To evaluate any method tackling this task, we developed an MSG dataset and annotation based on a public 3D dataset. We also propose an evaluation metric based on the intersection-over-union score of MSG edges. Moreover, we develop a novel baseline method built on mainstream pretrained vision models, combining visual place recognition and object association into one Transformer decoder architecture. Experiments demonstrate that our method has superior performance compared to existing relevant baselines.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multiview Scene Graph Generation on ARKitScenes (test)](<https://www.sota2.com/research/sota/multiview-scene-graph-generation-on-arkitscenes-test.md>): Multiview Scene Graph Generation; ARKitScenes (test); Recall@1: 98.61.

- [Multi-Scene Graph Construction on ARKitScenes 1.0 (test)](<https://www.sota2.com/research/sota/multi-scene-graph-construction-on-arkitscenes-1-0-test.md>): Multi-Scene Graph (MSG) Construction; ARKitScenes 1.0 (test); Recall@1: 98.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2410.11187>)

[PDF](<https://arxiv.org/pdf/2410.11187.pdf>)

[Implementation](<https://ai4ce.github.io/MSG/>)
