# RGBX-R1: Visual Modality Chain-of-Thought Guided Reinforcement Learning for Multimodal Grounding

Source: [RGBX-R1: Visual Modality Chain-of-Thought Guided Reinforcement Learning for Multimodal Grounding](<https://www.sota2.com/research/paper/69818100a8b01e76cc8e21f2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-04T05:00:04.388Z

Authors: Jiahe Wu, Bing Cao, Qilong Wang, Qinghua Hu, Dongdong Li, Pengfei Zhu

Published: 2026-01-31T04:13:57.000Z

## Abstract

Multimodal Large Language Models (MLLM) are primarily pre-trained on the RGB modality, thereby limiting their performance on other modalities, such as infrared, depth, and event data, which are crucial for complex scenarios. To address this, we propose RGBX-R1, a framework to enhance MLLM's perception and reasoning capacities across various X visual modalities. Specifically, we employ an Understand-Associate-Validate (UAV) prompting strategy to construct the Visual Modality Chain-of-Thought (VM-CoT), which aims to expand the MLLMs' RGB understanding capability into X modalities. To progressively enhance reasoning capabilities, we introduce a two-stage training paradigm: Cold-Start Supervised Fine-Tuning (CS-SFT) and Spatio-Temporal Reinforcement Fine-Tuning (ST-RFT). CS-SFT supervises the reasoning process with the guidance of VM-CoT, equipping the MLLM with fundamental modality cognition. Building upon GRPO, ST-RFT employs a Modality-understanding Spatio-Temporal (MuST) reward to reinforce modality reasoning. Notably, we construct the first RGBX-Grounding benchmark, and extensive experiments verify our superiority in multimodal understanding and spatial perception, outperforming baselines by 22.71% on three RGBX grounding tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Grounding on RGBX-Grounding](<https://www.sota2.com/research/sota/grounding-on-rgbx-grounding.md>): Grounding; RGBX-Grounding; LasHeR: 36.46.

- [Multi-image Grounding on RGBX-Grounding 1.0 (test)](<https://www.sota2.com/research/sota/multi-image-grounding-on-rgbx-grounding-1-0-test.md>): Multi-image Grounding; RGBX-Grounding 1.0 (test); RGBT234 Score: 47.27.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2602.00504>)

[PDF](<https://arxiv.org/pdf/2602.00504.pdf>)
