# Video Reasoning on MMMU Video Source: [Video Reasoning on MMMU Video]() Collection index: [llms.txt]() Data updated: 2026-03-05T07:09:30.392Z Evaluation of video reasoning capabilities using the MMMU Video dataset, measured by Accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Video Reasoning | | Dataset (including any stated split) | MMMU Video | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | GPT-5-thinking | 84.6 | Not recorded | 6926896bd6b0dd334293026f | | Gemini 2.5 Pro | 83.6 | Not recorded | 6926896bd6b0dd334293026f | | OpenAI O3 | 83.3 | Not recorded | 6926896bd6b0dd334293026f | | Seed 1.5VL | 81.4 | Not recorded | 6926896bd6b0dd334293026f | | Qwen3-VL-235B-Thinking | 80 | Not recorded | 6926896bd6b0dd334293026f | | VideoChat-M1 | 80 | Not recorded | 6926896bd6b0dd334293026f | | Qwen3-VL-235B-Instruct | 74.7 | Not recorded | 6926896bd6b0dd334293026f | | Qwen3-VL-8B-Instruct | 65.3 | Not recorded | 6926896bd6b0dd334293026f | | GPT-4o | 61.2 | Not recorded | 6926896bd6b0dd334293026f | | Gemini 1.5 Pro | 53.9 | Not recorded | 6926896bd6b0dd334293026f | | VideoChat-R1.5-7B | 51.4 | Not recorded | 6926896bd6b0dd334293026f | | Aria-28B | 50.8 | Not recorded | 6926896bd6b0dd334293026f | | LLAVA-Video-72B | 49.7 | Not recorded | 6926896bd6b0dd334293026f | | LLaVA-ov-72B | 48.3 | Not recorded | 6926896bd6b0dd334293026f | | LLaVA-Video-7B | 36.1 | Not recorded | 6926896bd6b0dd334293026f | | LongVA-7B | 24 | Not recorded | 6926896bd6b0dd334293026f | ### GPT-5-thinking Accuracy: 84.6. Result ID: 69a92c216dbb7075b9521243. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Large-sized MLLM"}. This record does not assert independent reproduction. ### Gemini 2.5 Pro Accuracy: 83.6. Result ID: 69a92c216dbb7075b9521241. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Large-sized MLLM"}. This record does not assert independent reproduction. ### OpenAI O3 Accuracy: 83.3. Result ID: 69a92c216dbb7075b9521245. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Large-sized MLLM"}. This record does not assert independent reproduction. ### Seed 1.5VL Accuracy: 81.4. Result ID: 69a92c216dbb7075b9521246. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Large-sized MLLM"}. This record does not assert independent reproduction. ### Qwen3-VL-235B-Thinking Accuracy: 80. Result ID: 69a92c216dbb7075b9521248. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-source Large-sized MLLM"}. This record does not assert independent reproduction. ### VideoChat-M1 Accuracy: 80. Result ID: 69a92c216dbb7075b9521250. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Ours","Number of Parameters":"37B"}. This record does not assert independent reproduction. ### Qwen3-VL-235B-Instruct Accuracy: 74.7. Result ID: 69a92c216dbb7075b9521247. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-source Large-sized MLLM"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-Instruct Accuracy: 65.3. Result ID: 69a92c216dbb7075b952124d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-source Medium-sized MLLM"}. This record does not assert independent reproduction. ### GPT-4o Accuracy: 61.2. Result ID: 69a92c216dbb7075b9521244. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Large-sized MLLM"}. This record does not assert independent reproduction. ### Gemini 1.5 Pro Accuracy: 53.9. Result ID: 69a92c216dbb7075b9521242. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Large-sized MLLM"}. This record does not assert independent reproduction. ### VideoChat-R1.5-7B Accuracy: 51.4. Result ID: 69a92c216dbb7075b952124c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-source Medium-sized MLLM"}. This record does not assert independent reproduction. ### Aria-28B Accuracy: 50.8. Result ID: 69a92c216dbb7075b952124b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Open-source Large-sized MLLM"}. This record does not assert independent reproduction. ### LLAVA-Video-72B Accuracy: 49.7. Result ID: 69a92c216dbb7075b9521249. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Open-source Large-sized MLLM"}. This record does not assert independent reproduction. ### LLaVA-ov-72B Accuracy: 48.3. Result ID: 69a92c216dbb7075b952124a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Open-source Large-sized MLLM"}. This record does not assert independent reproduction. ### LLaVA-Video-7B Accuracy: 36.1. Result ID: 69a92c216dbb7075b952124f. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Open-source Medium-sized MLLM"}. This record does not assert independent reproduction. ### LongVA-7B Accuracy: 24. Result ID: 69a92c216dbb7075b952124e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Open-source Medium-sized MLLM"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.