# Multimodal Reasoning on MMMU-Pro (Pass@1) Source: [Multimodal Reasoning on MMMU-Pro (Pass@1)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:57:30.267Z Evaluation of model performance on the MMMU-Pro dataset focusing on multimodal reasoning capabilities, measured specifically by the Pass@1 metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal reasoning | | Dataset (including any stated split) | MMMU-Pro | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | GPT-5-Nano-High | 70.64 | Not recorded | 6997eaa309c33900fcd7665d | | Qwen3-VL-8B-Thinking | 70.29 | Not recorded | 6997eaa309c33900fcd7665d | | Qwen3-VL-8B-DeepVision | 70.29 | Not recorded | 6997eaa309c33900fcd7665d | | MiMo-VL-7B-DeepVision | 69.19 | Not recorded | 6997eaa309c33900fcd7665d | | Gemini 1.5 Pro | 68.8 | Not recorded | 68fa78791e209131cea788a1 | | Qwen3-VL-8B-Instruct | 67.69 | Not recorded | 6997eaa309c33900fcd7665d | | Seed 1.5-VL | 67.6 | Not recorded | 68fa78791e209131cea788a1 | | MiMo-VL-7B-OpenMMReasoner | 66.82 | Not recorded | 6997eaa309c33900fcd7665d | | OpenAI o1 | 66.4 | Not recorded | 68fa78791e209131cea788a1 | | MiMo-VL-7B-MM-Eureka | 65.78 | Not recorded | 6997eaa309c33900fcd7665d | | Gemini-2.5-Flash-Lite | 65.08 | Not recorded | 6997eaa309c33900fcd7665d | | MiMo-VL-7B-RL-2508 | 63.87 | Not recorded | 6997eaa309c33900fcd7665d | | MiMo-VL-7B-MathBook | 63.47 | Not recorded | 6997eaa309c33900fcd7665d | | MiMo-VL-7B-SFT-2508 | 60.69 | Not recorded | 6997eaa309c33900fcd7665d | | Seed 1.5-VL | 59.9 | Not recorded | 68fa78791e209131cea788a1 | | GPT-4o | 54.5 | Not recorded | 68fa78791e209131cea788a1 | | Qwen 2.5-VL 72B | 51.1 | Not recorded | 68fa78791e209131cea788a1 | | Claude 3.7 Sonnet | 50.1 | Not recorded | 68fa78791e209131cea788a1 | ### GPT-5-Nano-High Pass@1: 70.64. Result ID: 699afe5a5698665ec4901cd5. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Models"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-Thinking Pass@1: 70.29. Result ID: 699afe5a5698665ec4901cd8. [Reporting paper]() Recorded evaluation configuration: {"Series":"Qwen3-VL-8B"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-DeepVision Pass@1: 70.29. Result ID: 699afe5a5698665ec4901cd9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Series":"Qwen3-VL-8B","Training Algorithm":"GSPO"}. This record does not assert independent reproduction. ### MiMo-VL-7B-DeepVision Pass@1: 69.19. Result ID: 699afe5a5698665ec4901cdf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Series":"MiMo-VL-7B","Training Algorithm":"GSPO"}. This record does not assert independent reproduction. ### Gemini 1.5 Pro Pass@1: 68.8. Result ID: 69958c593badfaffe685f902. [Reporting paper]() Recorded evaluation configuration: {"thinking":true,"decoding":"greedy"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-Instruct Pass@1: 67.69. Result ID: 699afe5a5698665ec4901cd7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Series":"Qwen3-VL-8B"}. This record does not assert independent reproduction. ### Seed 1.5-VL Pass@1: 67.6. Result ID: 69958c593badfaffe685f900. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"thinking":true,"decoding":"greedy"}. This record does not assert independent reproduction. ### MiMo-VL-7B-OpenMMReasoner Pass@1: 66.82. Result ID: 699afe5a5698665ec4901cde. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Series":"MiMo-VL-7B"}. This record does not assert independent reproduction. ### OpenAI o1 Pass@1: 66.4. Result ID: 69958c593badfaffe685f903. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"thinking":true,"decoding":"greedy"}. This record does not assert independent reproduction. ### MiMo-VL-7B-MM-Eureka Pass@1: 65.78. Result ID: 699afe5a5698665ec4901cdc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Series":"MiMo-VL-7B"}. This record does not assert independent reproduction. ### Gemini-2.5-Flash-Lite Pass@1: 65.08. Result ID: 699afe5a5698665ec4901cd6. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Closed-source Models"}. This record does not assert independent reproduction. ### MiMo-VL-7B-RL-2508 Pass@1: 63.87. Result ID: 699afe5a5698665ec4901cdb. [Reporting paper]() Recorded evaluation configuration: {"Series":"MiMo-VL-7B"}. This record does not assert independent reproduction. ### MiMo-VL-7B-MathBook Pass@1: 63.47. Result ID: 699afe5a5698665ec4901cdd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Series":"MiMo-VL-7B"}. This record does not assert independent reproduction. ### MiMo-VL-7B-SFT-2508 Pass@1: 60.69. Result ID: 699afe5a5698665ec4901cda. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Series":"MiMo-VL-7B"}. This record does not assert independent reproduction. ### Seed 1.5-VL Pass@1: 59.9. Result ID: 69958c593badfaffe685f901. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"thinking":false,"decoding":"greedy"}. This record does not assert independent reproduction. ### GPT-4o Pass@1: 54.5. Result ID: 69958c593badfaffe685f905. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"thinking":false,"decoding":"greedy"}. This record does not assert independent reproduction. ### Qwen 2.5-VL 72B Pass@1: 51.1. Result ID: 69958c593badfaffe685f906. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"thinking":false,"decoding":"greedy"}. This record does not assert independent reproduction. ### Claude 3.7 Sonnet Pass@1: 50.1. Result ID: 69958c593badfaffe685f904. [Reporting paper]() Recorded evaluation configuration: {"thinking":true,"decoding":"sampling"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.