# VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

Source: [VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs](<https://www.sota2.com/research/paper/692689a5d6b0dd33429304dd>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-08T05:00:04.940Z

Authors: Tianxiang Jiang, Sheng Xia, Yicheng Xu, Linquan Wu, Xiangyu Zeng, Limin Wang, Yu Qiao, Yi Wang

Published: 2025-11-25T12:58:32.000Z

## Abstract

While Multimodal Large Language Models (MLLMs) have become adept at recognizing objects, they often lack the intuitive, human-like understanding of the world's underlying physical and social principles. This high-level vision-grounded semantics, which we term visual knowledge, forms a bridge between perception and reasoning, yet remains an underexplored area in current MLLMs. To systematically evaluate this capability, we present VKnowU, a comprehensive benchmark featuring 1,680 questions in 1,249 videos, covering 8 core types of visual knowledge spanning both world-centric (e.g., intuitive physics) and human-centric (e.g., subjective intentions). Evaluation of 28 SOTA MLLMs reveals that leading models still fall short of human performance, with particularly notable gaps in the world-centric. To bridge this gap, we introduce a new dataset, VKnowQA, and VideoKnow+, a baseline model that explicitly incorporates visual knowledge into MLLMs. VideoKnow+ follows a structured See-Think-Answer paradigm and adopts reinforcement learning with visual knowledge reward, achieving a +3.7% improvement on VKnowU and consistent gains on MVBench (+5.4%), Video-MME (+7.0%), and MMVU (+5.7%). Our work highlights visual knowledge as a missing cornerstone for developing more generalizable MLLMs that can not only see but also truly understand our worlds.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Video Understanding on VKnowU](<https://www.sota2.com/research/sota/video-understanding-on-vknowu.md>): Video Understanding; VKnowU; Score: 67.7.

- [Multi-modal Visual Reasoning on VKnowU](<https://www.sota2.com/research/sota/multi-modal-visual-reasoning-on-vknowu.md>): Multi-modal Visual Reasoning; VKnowU; Overall Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.20272>)

[PDF](<https://arxiv.org/pdf/2511.20272.pdf>)