# MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chains

Source: [MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chains](<https://www.sota2.com/research/paper/68fa74701e209131cea61cdb>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:57:08.622Z

Authors: Zhaohuan Zhan, Lisha Yu, Sijie Yu, Guang Tan

Published: 2024-05-17T08:33:27.000Z

## Abstract

In the Vision-and-Language Navigation (VLN) task, the agent is required to navigate to a destination following a natural language instruction. While learning-based approaches have been a major solution to the task, they suffer from high training costs and lack of interpretability. Recently, Large Language Models (LLMs) have emerged as a promising tool for VLN due to their strong generalization capabilities. However, existing LLM-based methods face limitations in memory construction and diversity of navigation strategies. To address these challenges, we propose a suite of techniques. Firstly, we introduce a method to maintain a topological map that stores navigation history, retaining information about viewpoints, objects, and their spatial relationships. This map also serves as a global action space. Additionally, we present a Navigation Chain of Thoughts module, leveraging human navigation examples to enrich navigation strategy diversity. Finally, we establish a pipeline that integrates navigational memory and strategies with perception and action prediction modules. Experimental results on the REVERIE and R2R datasets show that our method effectively enhances the navigation ability of the LLM and improves the interpretability of navigation reasoning.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Vision-and-Language Navigation on REVERIE Unseen Discrete (val)](<https://www.sota2.com/research/sota/vision-and-language-navigation-on-reverie-unseen-discrete-val.md>): Vision-and-Language Navigation; REVERIE Unseen Discrete (val); OSR: 30.3.

- [Vision-and-Language Navigation on R2R Discrete (val-unseen)](<https://www.sota2.com/research/sota/vision-and-language-navigation-on-r2r-discrete-val-unseen.md>): Vision-and-Language Navigation; R2R Discrete (val-unseen); Navigation Error (NE): 5.42.

- [Vision-and-Language Navigation on R2R (val unseen)](<https://www.sota2.com/research/sota/vision-and-language-navigation-on-r2r-val-unseen.md>): Vision-and-Language Navigation; R2R (val unseen); Success Rate (SR): 32.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2405.10620>)

[PDF](<https://arxiv.org/pdf/2405.10620.pdf>)