# Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch? Source: [Can MLLMs Absorb Math Reasoning Abilities from LLMs as Free Lunch?]() Collection index: [llms.txt]() Data updated: 2025-10-26T05:58:13.123Z Authors: Yijie Hu, Zihao Zhou, Kaizhu Huang, Xiaowei Huang, Qiufeng Wang Published: 2025-10-16T07:38:16.000Z ## Abstract Math reasoning has been one crucial ability of large language models (LLMs), where significant advancements have been achieved in recent years. However, most efforts focus on LLMs by curating high-quality annotation data and intricate training (or inference) paradigms, while the math reasoning performance of multi-modal LLMs (MLLMs) remains lagging behind. Since the MLLM typically consists of an LLM and a vision block, we wonder: Can MLLMs directly absorb math reasoning abilities from off-the-shelf math LLMs without tuning? Recent model-merging approaches may offer insights into this question. However, they overlook the alignment between the MLLM and LLM, where we find that there is a large gap between their parameter spaces, resulting in lower performance. Our empirical evidence reveals two key factors behind this issue: the identification of crucial reasoning-associated layers in the model and the mitigation of the gaps in parameter space. Based on the empirical insights, we propose IP-Merging that first identifies the reasoning-associated parameters in both MLLM and Math LLM, then projects them into the subspace of MLLM, aiming to maintain the alignment, and finally merges parameters in this subspace. IP-Merging is a tuning-free approach since parameters are directly adjusted. Extensive experiments demonstrate that our IP-Merging method can enhance the math reasoning ability of MLLMs directly from Math LLMs without compromising their other capabilities. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Vision-Language Reasoning on VL Reasoning Benchmarks](): Vision-Language Reasoning; VL Reasoning Benchmarks; MVista Score: 73.4. - [Vision-Language Reasoning on VL Reasoning Suite (MathVista, MMMU, MMStar)](): Vision-Language Reasoning; VL Reasoning Benchmarks MathVista, MVerse, MathVision, MMMU, R1-OV, MMStar; MathVista Acc: 75.5. - [Vision-Language Perception on VL Perception Benchmarks TextVQA, POPE, Seed-Bench](): Vision-Language Perception; VL Perception Benchmarks TextVQA, POPE, Seed-Bench; TextVQA Score: 82.9. - [Vision-Language Perception on VL Perception Benchmarks](): Vision-Language Perception; VL Perception Benchmarks; TextVQA: 84.8. ## Original sources and implementation [Original paper]() [PDF]()