# Grounding Complex Navigational Instructions Using Scene Graphs

Source: [Grounding Complex Navigational Instructions Using Scene Graphs](<https://www.sota2.com/research/paper/68fa6a0c1e209131cea25b84>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:15:37.395Z

Authors: Michiel de Jong, Satyapriya Krishna, Anuva Agarwal

Published: 2021-06-03T05:45:21.000Z

## Abstract

Training a reinforcement learning agent to carry out natural language instructions is limited by the available supervision, i.e. knowing when the instruction has been carried out. We adapt the CLEVR visual question answering dataset to generate complex natural language navigation instructions and accompanying scene graphs, yielding an environment-agnostic supervised dataset. To demonstrate the use of this data set, we map the scenes to the VizDoom environment and use the architecture in \\citet{gatedattention} to train an agent to carry out these more complex language instructions.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2106.01607>)

[PDF](<https://arxiv.org/pdf/2106.01607.pdf>)