# Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation

Source: [Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation](<https://www.sota2.com/research/paper/6a337b96206542251d7c50d7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-24T05:00:04.550Z

Authors: Sihan Wang, Xiyao Liu, Lianqing Liu, Zhi Han

Published: 2026-06-17T14:33:38.000Z

## Abstract

On-policy self-distillation (OPSD) trains a model on its own rollouts and uses a frozen copy to provide dense token-level targets conditioned on a reference target. This works well for LLM reasoning, but a direct extension to multimodal large language models (MLLMs) can create a shortcut: the privileged target may guide tokens mainly based on the text reference target rather than the image. We propose ViGOS, a visually grounded OPSD framework for MLLM post-training. The student first writes a visual description and then reasons toward the final answer. For valid rollouts, an image-only perception teacher supervises the description, while a privileged reasoning teacher supervises the reasoning and final answer on the same student prefix. A reference teacher is used only for invalid rollouts to recover the output format. Across general vision-language, expert reasoning, visual math, spatial grounding, and visual-language-prior benchmarks, ViGOS keeps the main benefits of OPSD and improves image-grounded behavior in shortcut-prone settings.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Expert/Academic Reasoning on MMMU (Pass@5, Avg@5)](<https://www.sota2.com/research/sota/expert-academic-reasoning-on-mmmu-pass-at-5-avg-at-5.md>): Expert/Academic Reasoning; MMMU; Pass@5 Score: 80.11.

- [Expert/Academic Reasoning on MMMU-Pro (Pass@5, Avg@5)](<https://www.sota2.com/research/sota/expert-academic-reasoning-on-mmmu-pro-pass-at-5-avg-at-5.md>): Expert/Academic Reasoning; MMMU-Pro; Pass@5: 64.81.

- [Prior-Sensitive Evaluation on ViLP-F with-fact](<https://www.sota2.com/research/sota/prior-sensitive-evaluation-on-vilp-f-with-fact.md>): Prior-Sensitive Evaluation; ViLP-F with-fact; Score: 70.17.

- [Prior-Sensitive Evaluation on ViLP-P pure-question](<https://www.sota2.com/research/sota/prior-sensitive-evaluation-on-vilp-p-pure-question.md>): Prior-Sensitive Evaluation; ViLP-P pure-question; Score: 69.5.

- [Spatial & Vision Grounding on MMSI](<https://www.sota2.com/research/sota/spatial-and-vision-grounding-on-mmsi.md>): Spatial & Vision Grounding; MMSI; Pass@5: 66.4.

- [Spatial & Vision Grounding on CV-Bench](<https://www.sota2.com/research/sota/spatial-and-vision-grounding-on-cv-bench.md>): Spatial & Vision Grounding; CV-Bench; Pass@5: 91.51.

- [Visual Math on MathVista](<https://www.sota2.com/research/sota/visual-math-on-mathvista.md>): Visual Math; MathVista; Pass@5: 80.9.

- [Visual Math on MathVerse](<https://www.sota2.com/research/sota/visual-math-on-mathverse.md>): Visual Math; MathVerse; Pass@5: 68.91.

- [General Visual Language Performance on MM-Vet (Pass@5)](<https://www.sota2.com/research/sota/general-visual-language-performance-on-mm-vet-pass-at-5.md>): General Visual Language; MM-Vet; Pass@5: 72.02.

- [Spatial & Vision Grounding on RealWorldQA](<https://www.sota2.com/research/sota/spatial-and-vision-grounding-on-realworldqa.md>): Spatial & Vision Grounding; RealworldQA; Pass@5: 86.8.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.19120>)

[PDF](<https://arxiv.org/pdf/2606.19120.pdf>)

[Implementation](<https://github.com/OedoSoldier/ViGOS>)
