# Video Understanding and Reasoning on Video-MMMU (test) Source: [Video Understanding and Reasoning on Video-MMMU (test)]() Collection index: [llms.txt]() Data updated: 2026-04-23T10:50:22.952Z Evaluation of general video understanding and reasoning capabilities on the test split of the Video-MMMU dataset, reporting scores for Overall, Perception, Comprehension, and Adaptation. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Video Understanding and Reasoning | | Dataset (including any stated split) | Video-MMMU (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 44 of 44 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Overall Score](): Higher is better; unit: PERCENT. - [Perception Score](): Higher is better; unit: PERCENT. - [Comprehension Score](): Higher is better; unit: PERCENT. - [Adaptation Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | GPT-4o | 0.612 | Not recorded | 69e9a78555a3c32af89f6feb | | VIDEOP2R | 0.55 | Not recorded | 691aabf76750be83fe5ce1d7 | | VideoChat-R1 | 0.54 | Not recorded | 691aabf76750be83fe5ce1d7 | | Video-R1 | 0.524 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-R1 | 0.523 | Not recorded | 691aabf76750be83fe5ce1d7 | | Video-R1 | 0.523 | Not recorded | 69e9a78555a3c32af89f6feb | | VersaVid-R1 | 0.519 | Not recorded | 691aabf76750be83fe5ce1d7 | | Video-ToC | 0.513 | Not recorded | 69e9a78555a3c32af89f6feb | | VideoRFT | 0.511 | Not recorded | 691aabf76750be83fe5ce1d7 | | Time-R1 | 0.51 | Not recorded | 691aabf76750be83fe5ce1d7 | | Video-ToC | 0.51 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-ToC | 0.505 | Not recorded | 69e9a78555a3c32af89f6feb | | Baseline (Qwen2.5-VL-7B) | 0.504 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-R1 | 0.498 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-R1-SFT | 0.494 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-R1-SFT | 0.494 | Not recorded | 69e9a78555a3c32af89f6feb | | VideoChat-R1 | 0.487 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-ToC-SFT | 0.483 | Not recorded | 69e9a78555a3c32af89f6feb | | Qwen2.5-VL-7B | 0.481 | Not recorded | 691aabf76750be83fe5ce1d7 | | Baseline (Qwen2.5-VL-7B) | 0.481 | Not recorded | 69e9a78555a3c32af89f6feb | | Baseline (Qwen2.5-VL-7B) | 0.478 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-R1-SFT | 0.474 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-ToC-SFT | 0.471 | Not recorded | 69e9a78555a3c32af89f6feb | | Video-ToC-SFT | 0.465 | Not recorded | 69e9a78555a3c32af89f6feb | | LLaVA-OneVision-7B | 0.338 | Not recorded | 691aabf76750be83fe5ce1d7 | | LLaVA-OneVision-7B | 0.338 | Not recorded | 69e9a78555a3c32af89f6feb | | LongVA-7B | 0.239 | Not recorded | 691aabf76750be83fe5ce1d7 | | LongVA-7B | 0.239 | Not recorded | 69e9a78555a3c32af89f6feb | | VILA-1.5-8B | 0.208 | Not recorded | 69e9a78555a3c32af89f6feb | ### GPT-4o Accuracy: 0.612. Result ID: 69e9f95447f34ce924d1f50e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"-"}. This record does not assert independent reproduction. ### VIDEOP2R Accuracy: 0.55. Result ID: 69e75cfd47f34ce924ce9295. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### VideoChat-R1 Accuracy: 0.54. Result ID: 69e75cfd47f34ce924ce9291. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### Video-R1 Accuracy: 0.524. Result ID: 69e9f95447f34ce924d1f520. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"64"}. This record does not assert independent reproduction. ### Video-R1 Accuracy: 0.523. Result ID: 69e75cfd47f34ce924ce9290. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### Video-R1 Accuracy: 0.523. Result ID: 69e9f95447f34ce924d1f51b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"32"}. This record does not assert independent reproduction. ### VersaVid-R1 Accuracy: 0.519. Result ID: 69e75cfd47f34ce924ce9293. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### Video-ToC Accuracy: 0.513. Result ID: 69e9f95447f34ce924d1f51c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"32"}. This record does not assert independent reproduction. ### VideoRFT Accuracy: 0.511. Result ID: 69e75cfd47f34ce924ce9294. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### Time-R1 Accuracy: 0.51. Result ID: 69e75cfd47f34ce924ce9292. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B"}. This record does not assert independent reproduction. ### Video-ToC Accuracy: 0.51. Result ID: 69e9f95447f34ce924d1f521. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"64"}. This record does not assert independent reproduction. ### Video-ToC Accuracy: 0.505. Result ID: 69e9f95447f34ce924d1f517. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"16"}. This record does not assert independent reproduction. ### Baseline (Qwen2.5-VL-7B) Accuracy: 0.504. Result ID: 69e9f95447f34ce924d1f51d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"64"}. This record does not assert independent reproduction. ### Video-R1 Accuracy: 0.498. Result ID: 69e9f95447f34ce924d1f516. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"16"}. This record does not assert independent reproduction. ### Video-R1-SFT Accuracy: 0.494. Result ID: 69e9f95447f34ce924d1f519. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"32"}. This record does not assert independent reproduction. ### Video-R1-SFT Accuracy: 0.494. Result ID: 69e9f95447f34ce924d1f51e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"64"}. This record does not assert independent reproduction. ### VideoChat-R1 Accuracy: 0.487. Result ID: 69e9f95447f34ce924d1f515. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"16"}. This record does not assert independent reproduction. ### Video-ToC-SFT Accuracy: 0.483. Result ID: 69e9f95447f34ce924d1f51f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"64"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B Accuracy: 0.481. Result ID: 69e75cfd47f34ce924ce928f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B"}. This record does not assert independent reproduction. ### Baseline (Qwen2.5-VL-7B) Accuracy: 0.481. Result ID: 69e9f95447f34ce924d1f518. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"32"}. This record does not assert independent reproduction. ### Baseline (Qwen2.5-VL-7B) Accuracy: 0.478. Result ID: 69e9f95447f34ce924d1f512. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"16"}. This record does not assert independent reproduction. ### Video-R1-SFT Accuracy: 0.474. Result ID: 69e9f95447f34ce924d1f513. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"16"}. This record does not assert independent reproduction. ### Video-ToC-SFT Accuracy: 0.471. Result ID: 69e9f95447f34ce924d1f51a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"32"}. This record does not assert independent reproduction. ### Video-ToC-SFT Accuracy: 0.465. Result ID: 69e9f95447f34ce924d1f514. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Frames":"16"}. This record does not assert independent reproduction. ### LLaVA-OneVision-7B Accuracy: 0.338. Result ID: 69e75cfd47f34ce924ce928d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"7B"}. This record does not assert independent reproduction. ### LLaVA-OneVision-7B Accuracy: 0.338. Result ID: 69e9f95447f34ce924d1f511. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Frames":"-"}. This record does not assert independent reproduction. ### LongVA-7B Accuracy: 0.239. Result ID: 69e75cfd47f34ce924ce928e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Scale":"7B"}. This record does not assert independent reproduction. ### LongVA-7B Accuracy: 0.239. Result ID: 69e9f95447f34ce924d1f50f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Frames":"-"}. This record does not assert independent reproduction. ### VILA-1.5-8B Accuracy: 0.208. Result ID: 69e9f95447f34ce924d1f510. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Frames":"-"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.