# Multimodal Reasoning on MMK12 (Pass@1) Source: [Multimodal Reasoning on MMK12 (Pass@1)]() Collection index: [llms.txt]() Data updated: 2026-05-13T09:14:07.549Z Evaluation of model performance on the MMK12 dataset for Multimodal Reasoning capabilities, measured using the Pass@1 metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Reasoning | | Dataset (including any stated split) | MMK12 | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | Masking-KD-4B | 51 | Not recorded | 6a0405c4dbc836b87d82779f | | Self-distill Masking-KD-8B | 49.95 | Not recorded | 6a0405c4dbc836b87d82779f | | Ovis2-8B | 48.15 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3.5-8B | 44.95 | Not recorded | 6a0405c4dbc836b87d82779f | | MiMo-VL-8B | 44.1 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen2.5-VL-7B | 43.9 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen3-VL-8B-Thinking | 42.55 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen2.5-VL-3B | 39.85 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3-8B | 39.8 | Not recorded | 6a0405c4dbc836b87d82779f | | Ovis2-4B | 39.1 | Not recorded | 6a0405c4dbc836b87d82779f | | Masking-KD-2B | 37.2 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3-2B | 37 | Not recorded | 6a0405c4dbc836b87d82779f | | Ovis2-2B | 32.45 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen3-VL-4B-Thinking | 31.55 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3.5-4B | 25.8 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3.5-2B | 14.7 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen3-VL-2B-Thinking | 13 | Not recorded | 6a0405c4dbc836b87d82779f | ### Masking-KD-4B Pass@1: 51. Result ID: 6a043fef47f34ce924ea5340. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096","Distillation":"Distilled from 8B"}. This record does not assert independent reproduction. ### Self-distill Masking-KD-8B Pass@1: 49.95. Result ID: 6a043fef47f34ce924ea533b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096","Distillation":"Self-distill"}. This record does not assert independent reproduction. ### Ovis2-8B Pass@1: 48.15. Result ID: 6a043fef47f34ce924ea5336. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3.5-8B Pass@1: 44.95. Result ID: 6a043fef47f34ce924ea5337. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### MiMo-VL-8B Pass@1: 44.1. Result ID: 6a043fef47f34ce924ea5339. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B Pass@1: 43.9. Result ID: 6a043fef47f34ce924ea533a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-Thinking Pass@1: 42.55. Result ID: 6a043fef47f34ce924ea5335. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen2.5-VL-3B Pass@1: 39.85. Result ID: 6a043fef47f34ce924ea533f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3-8B Pass@1: 39.8. Result ID: 6a043fef47f34ce924ea5338. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Ovis2-4B Pass@1: 39.1. Result ID: 6a043fef47f34ce924ea533d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Masking-KD-2B Pass@1: 37.2. Result ID: 6a043fef47f34ce924ea5345. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096","Distillation":"Distilled from 8B"}. This record does not assert independent reproduction. ### InternVL3-2B Pass@1: 37. Result ID: 6a043fef47f34ce924ea5344. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Ovis2-2B Pass@1: 32.45. Result ID: 6a043fef47f34ce924ea5342. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen3-VL-4B-Thinking Pass@1: 31.55. Result ID: 6a043fef47f34ce924ea533c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3.5-4B Pass@1: 25.8. Result ID: 6a043fef47f34ce924ea533e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3.5-2B Pass@1: 14.7. Result ID: 6a043fef47f34ce924ea5343. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen3-VL-2B-Thinking Pass@1: 13. Result ID: 6a043fef47f34ce924ea5341. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.