# Multimodal Continual Learning with MLLMs from Multi-scenario Perspectives

Source: [Multimodal Continual Learning with MLLMs from Multi-scenario Perspectives](<https://www.sota2.com/research/paper/69253816d6b0dd334292f46e>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-17T05:00:04.157Z

Authors: Kai Jiang, Siqi Huang, Xiangyu Chen, Jiawei Shao, Hongyuan Zhang, Ping Luo, Xuelong Li

Published: 2025-11-23T15:47:49.000Z

## Abstract

Multimodal large language models (MLLMs) deployed on devices must adapt to continuously changing visual scenarios such as variations in background and perspective, to effectively perform complex visual tasks. To investigate catastrophic forgetting under real-world scenario shifts, we construct a multimodal visual understanding dataset (MSVQA), covering four distinct scenarios and perspectives: high-altitude, underwater, low-altitude, and indoor environments. Furthermore, we propose UNIFIER (mUltimodal coNtInual learning with MLLMs From multi-scenarIo pERspectives), a continual learning (CL) framework designed to address visual discrepancies while learning different scenarios. Compared to existing CL methods, UNIFIER enables knowledge accumulation within the same scenario and mutual enhancement across different scenarios via Vision Representation Expansion (VRE) and Vision Consistency Constraint (VCC). Experimental results show that UNIFIER improves the last-step VQA scores by 2.70%~10.62% and the last-step F1 scores by 3.40%~7.69% compared to the state-of-the-art method, QUAD, in 20-step cross-scenario continual learning tasks. MSVQA dataset is available at https://huggingface.co/datasets/Kaij00/MSVQA.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on MSVQA High altitude T=5 (test)](<https://www.sota2.com/research/sota/object-detection-on-msvqa-high-altitude-t-5-test.md>): Object Detection; MSVQA High altitude T=5 (test); Last Performance (At): 73.76.

- [Object Detection on MSVQA Underwater T=5 (test)](<https://www.sota2.com/research/sota/object-detection-on-msvqa-underwater-t-5-test.md>): Object Detection; MSVQA Underwater T=5 (test); Last Performance (At): 72.07.

- [Object Detection on MSVQA Low altitude T=5 (test)](<https://www.sota2.com/research/sota/object-detection-on-msvqa-low-altitude-t-5-test.md>): Object Detection; MSVQA Low altitude T=5 (test); At (Last Performance): 33.16.

- [Object Detection on MSVQA Indoor T=5 (test)](<https://www.sota2.com/research/sota/object-detection-on-msvqa-indoor-t-5-test.md>): Object Detection; MSVQA Indoor T=5 (test); Last Performance (At): 69.85.

- [Visual Question Answering on MSVQA High altitude T=5 (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-msvqa-high-altitude-t-5-test.md>): Visual Question Answering; MSVQA High altitude T=5 (test); Last Performance (At): 63.13.

- [Visual Question Answering on MSVQA Underwater T=5 (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-msvqa-underwater-t-5-test.md>): Visual Question Answering; MSVQA Underwater T=5 (test); Last Performance (At): 82.2.

- [Visual Question Answering on MSVQA Low altitude T=5 (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-msvqa-low-altitude-t-5-test.md>): Visual Question Answering; MSVQA Low altitude T=5 (test); Last Performance (At): 57.81.

- [Visual Question Answering on MSVQA Indoor T=5 (test)](<https://www.sota2.com/research/sota/visual-question-answering-on-msvqa-indoor-t-5-test.md>): Visual Question Answering; MSVQA Indoor T=5 (test); Last Performance (At): 85.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.18507>)

[PDF](<https://arxiv.org/pdf/2511.18507.pdf>)