# Visual Reasoning on REASONMAP Short questions Source: [Visual Reasoning on REASONMAP Short questions]() Collection index: [llms.txt]() Data updated: 2026-02-26T06:08:45.462Z Evaluation of model performance on the REASONMAP Short questions dataset for Visual Reasoning capabilities. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Visual Reasoning | | Dataset (including any stated split) | REASONMAP Short questions | | Metric | Weighted Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Weighted Accuracy](): Higher is better; unit: PERCENT. - [Weighted mAP Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Weighted Accuracy | Result date | Report paper | | --- | --- | --- | --- | | GPT-5 | 0.5998 | Not recorded | 68fa7bd61e209131cea8ad0a | | GPT-4o | 0.4115 | Not recorded | 68fa7bd61e209131cea8ad0a | | Seed1.5-VL | 0.342 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct + REWARDMAP | 0.3151 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct + RL (baseline) | 0.2951 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct + SFT -\> RL | 0.2882 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct + RL (REINFORCE++) | 0.2717 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-72B-Instruct | 0.2665 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct + RL (GRPO) | 0.2622 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct + RL (ReMax) | 0.2622 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-32B-Instruct | 0.1649 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct + SFT | 0.1363 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-7B-Instruct | 0.1328 | Not recorded | 68fa7bd61e209131cea8ad0a | | Kimi-VL-A3B-Instruct | 0.1276 | Not recorded | 68fa7bd61e209131cea8ad0a | | Qwen2.5-VL-3B-Instruct | 0.0868 | Not recorded | 68fa7bd61e209131cea8ad0a | | Kimi-VL-A3B-Thinking | 0.0547 | Not recorded | 68fa7bd61e209131cea8ad0a | ### GPT-5 Weighted Accuracy: 0.5998. Result ID: 699e1f975698665ec4d933a1. [Reporting paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### GPT-4o Weighted Accuracy: 0.4115. Result ID: 699e1f975698665ec4d933a0. [Reporting paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### Seed1.5-VL Weighted Accuracy: 0.342. Result ID: 699e1f975698665ec4d9339f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct + REWARDMAP Weighted Accuracy: 0.3151. Result ID: 699e1f975698665ec4d933a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":"RPlustrain + Rtrain"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct + RL (baseline) Weighted Accuracy: 0.2951. Result ID: 699e1f975698665ec4d933a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":"RPlustrain + Rtrain"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct + SFT -\> RL Weighted Accuracy: 0.2882. Result ID: 699e1f975698665ec4d933a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":"RPlustrain + Rtrain"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct + RL (REINFORCE++) Weighted Accuracy: 0.2717. Result ID: 699e1f975698665ec4d933a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":"Rtrain"}. This record does not assert independent reproduction. ### Qwen2.5-VL-72B-Instruct Weighted Accuracy: 0.2665. Result ID: 699e1f975698665ec4d9339e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct + RL (GRPO) Weighted Accuracy: 0.2622. Result ID: 699e1f975698665ec4d933a3. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training Data":"Rtrain"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct + RL (ReMax) Weighted Accuracy: 0.2622. Result ID: 699e1f975698665ec4d933a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":"Rtrain"}. This record does not assert independent reproduction. ### Qwen2.5-VL-32B-Instruct Weighted Accuracy: 0.1649. Result ID: 699e1f975698665ec4d9339d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct + SFT Weighted Accuracy: 0.1363. Result ID: 699e1f975698665ec4d933a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":"RPlustrain"}. This record does not assert independent reproduction. ### Qwen2.5-VL-7B-Instruct Weighted Accuracy: 0.1328. Result ID: 699e1f975698665ec4d933a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### Kimi-VL-A3B-Instruct Weighted Accuracy: 0.1276. Result ID: 699e1f975698665ec4d9339b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### Qwen2.5-VL-3B-Instruct Weighted Accuracy: 0.0868. Result ID: 699e1f975698665ec4d9339c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. ### Kimi-VL-A3B-Thinking Weighted Accuracy: 0.0547. Result ID: 699e1f975698665ec4d9339a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Training Data":null}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.