# From USD Scenes to Knowledge Graphs: Zero-Shot Ontology Grounding with LLMs

Source: [From USD Scenes to Knowledge Graphs: Zero-Shot Ontology Grounding with LLMs](<https://www.sota2.com/research/paper/6a279e5390966ade919bf9b0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-10T05:15:22.625Z

Authors: Jiangtao Shuai, Zongxiong Chen, Manfred Hauswirth, Sonja Schimmler

Published: 2026-06-08T07:32:06.000Z

## Abstract

Constructing knowledge graphs from 3D simulation scenes is essential for robot task reasoning, but the key bottleneck, grounding scene objects to formal ontology classes, still relies on manually curated dictionaries that are brittle and do not generalize across assets. We investigate whether large language models (LLMs) can automate this grounding step for Universal Scene Description (USD) scenes as a zero-shot, training-free alternative. On a kitchen scene (125 objects) with SOMA-HOME Ontology, LLMs achieve 90-96% exact-match accuracy with descriptive names and 49-89% with abbreviated names, substantially outperforming dictionary and embedding baselines. Under fully opaque names, context-augmented prompting recovers up to 48%. Feature ablation reveals that LLMs primarily exploit semantic cues in the scene graph (sibling names and parent paths); anonymizing these cues reduces accuracy to 0-6%, while geometry alone yields only 4-17%.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Ontology Grounding on Pixar Kitchen Set semantic regime SOMA-HOME ontology (test)](<https://www.sota2.com/research/sota/ontology-grounding-on-pixar-kitchen-set-semantic-regime-soma-home-ontology-test.md>): Ontology Grounding; Pixar Kitchen Set semantic regime SOMA-HOME ontology (test); EMA: 96.

- [Ontology Grounding on Pixar Kitchen Set abbreviated regime SOMA-HOME ontology (test)](<https://www.sota2.com/research/sota/ontology-grounding-on-pixar-kitchen-set-abbreviated-regime-soma-home-ontology-test.md>): Ontology Grounding; Pixar Kitchen Set abbreviated regime SOMA-HOME ontology (test); EMA: 89.

- [Ontology Grounding on Pixar Kitchen Set opaque regime SOMA-HOME ontology (test)](<https://www.sota2.com/research/sota/ontology-grounding-on-pixar-kitchen-set-opaque-regime-soma-home-ontology-test.md>): Ontology Grounding; Pixar Kitchen Set opaque regime SOMA-HOME ontology (test); EMA: 48.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.09134>)

[PDF](<https://arxiv.org/pdf/2606.09134.pdf>)
