# Multimodal Reasoning on Geo3k Source: [Multimodal Reasoning on Geo3k]() Collection index: [llms.txt]() Data updated: 2026-05-13T09:14:07.466Z Evaluation of model performance on the Geo3k dataset for Multimodal Reasoning tasks, measured by Pass@1. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Multimodal Reasoning | | Dataset (including any stated split) | Geo3k | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 17 of 17 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | MiMo-VL-8B | 62.23 | Not recorded | 6a0405c4dbc836b87d82779f | | Self-distill Masking-KD-8B | 58.24 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen3-VL-8B-Thinking | 54.58 | Not recorded | 6a0405c4dbc836b87d82779f | | Masking-KD-4B | 52.58 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3.5-8B | 44.59 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen3-VL-4B-Thinking | 43.93 | Not recorded | 6a0405c4dbc836b87d82779f | | Ovis2-8B | 42.43 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3.5-4B | 41.93 | Not recorded | 6a0405c4dbc836b87d82779f | | Masking-KD-2B | 40.93 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen2.5-VL-7B | 40.43 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3-8B | 38.44 | Not recorded | 6a0405c4dbc836b87d82779f | | Ovis2-4B | 37.77 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3-2B | 33.78 | Not recorded | 6a0405c4dbc836b87d82779f | | Ovis2-2B | 31.11 | Not recorded | 6a0405c4dbc836b87d82779f | | InternVL3.5-2B | 29.95 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen2.5-VL-3B | 26.29 | Not recorded | 6a0405c4dbc836b87d82779f | | Qwen3-VL-2B-Thinking | 26.29 | Not recorded | 6a0405c4dbc836b87d82779f | ### MiMo-VL-8B Pass@1: 62.23. Result ID: 6a043fef47f34ce924ea5306. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Self-distill Masking-KD-8B Pass@1: 58.24. Result ID: 6a043fef47f34ce924ea5308. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096","Distillation":"Self-distill"}. This record does not assert independent reproduction. ### Qwen3-VL-8B-Thinking Pass@1: 54.58. Result ID: 6a043fef47f34ce924ea5302. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Masking-KD-4B Pass@1: 52.58. Result ID: 6a043fef47f34ce924ea530d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096","Distillation":"Distilled from 8B"}. This record does not assert independent reproduction. ### InternVL3.5-8B Pass@1: 44.59. Result ID: 6a043fef47f34ce924ea5304. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen3-VL-4B-Thinking Pass@1: 43.93. Result ID: 6a043fef47f34ce924ea5309. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Ovis2-8B Pass@1: 42.43. Result ID: 6a043fef47f34ce924ea5303. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3.5-4B Pass@1: 41.93. Result ID: 6a043fef47f34ce924ea530b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Masking-KD-2B Pass@1: 40.93. Result ID: 6a043fef47f34ce924ea5312. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096","Distillation":"Distilled from 8B"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B Pass@1: 40.43. Result ID: 6a043fef47f34ce924ea5307. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3-8B Pass@1: 38.44. Result ID: 6a043fef47f34ce924ea5305. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~8B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Ovis2-4B Pass@1: 37.77. Result ID: 6a043fef47f34ce924ea530a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3-2B Pass@1: 33.78. Result ID: 6a043fef47f34ce924ea5311. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Ovis2-2B Pass@1: 31.11. Result ID: 6a043fef47f34ce924ea530f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### InternVL3.5-2B Pass@1: 29.95. Result ID: 6a043fef47f34ce924ea5310. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen2.5-VL-3B Pass@1: 26.29. Result ID: 6a043fef47f34ce924ea530c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~4B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. ### Qwen3-VL-2B-Thinking Pass@1: 26.29. Result ID: 6a043fef47f34ce924ea530e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Size":"~2B","Decoding Strategy":"greedy","Max Length":"4096"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.