# Vision-Language Reasoning on VL Reasoning Suite (MathVista, MMMU, MMStar) Source: [Vision-Language Reasoning on VL Reasoning Suite (MathVista, MMMU, MMStar)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:51:39.659Z Comprehensive evaluation of model performance on a suite of Vision-Language Reasoning benchmarks, including MathVista, MVerse, MathVision, MMMU, R1-OV, and MMStar. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Vision-Language Reasoning | | Dataset (including any stated split) | VL Reasoning Benchmarks MathVista, MVerse, MathVision, MMMU, R1-OV, MMStar | | Metric | MathVista Acc | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [MathVista Acc](): Higher is better; unit: PERCENT. - [MVerse Acc](): Higher is better; unit: PERCENT. - [MathVision Acc](): Higher is better; unit: PERCENT. - [MMMU Acc](): Higher is better; unit: PERCENT. - [R1-OV Acc](): Higher is better; unit: PERCENT. - [MMStar Acc](): Higher is better; unit: PERCENT. - [Average Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | MathVista Acc | Result date | Report paper | | --- | --- | --- | --- | | FRISM | 79.8 | Not recorded | 697c3aefa8b01e76cc8da38e | | TA | 78.1 | Not recorded | 697c3aefa8b01e76cc8da38e | | Base | 77.7 | Not recorded | 697c3aefa8b01e76cc8da38e | | IP-Merging | 75.5 | Not recorded | 697c3aefa8b01e76cc8da38e | | Ties-Merging | 74.8 | Not recorded | 697c3aefa8b01e76cc8da38e | | FRISM | 70.3 | Not recorded | 697c3aefa8b01e76cc8da38e | | IP-Merging | 68.8 | Not recorded | 697c3aefa8b01e76cc8da38e | | DARE | 68.4 | Not recorded | 697c3aefa8b01e76cc8da38e | | Base | 68.1 | Not recorded | 697c3aefa8b01e76cc8da38e | | IP-Merging | 67.2 | Not recorded | 697c3aefa8b01e76cc8da38e | | IP-Merging | 65.9 | Not recorded | 697c3aefa8b01e76cc8da38e | | TA | 65.4 | Not recorded | 697c3aefa8b01e76cc8da38e | | Ties-Merging | 61.9 | Not recorded | 697c3aefa8b01e76cc8da38e | | FRISM | 45 | Not recorded | 697c3aefa8b01e76cc8da38e | | Ties-Merging | 41.8 | Not recorded | 697c3aefa8b01e76cc8da38e | | TA | 41.5 | Not recorded | 697c3aefa8b01e76cc8da38e | | TA | 41.4 | Not recorded | 697c3aefa8b01e76cc8da38e | | IP-Merging | 41.1 | Not recorded | 697c3aefa8b01e76cc8da38e | | IP-Merging | 41.1 | Not recorded | 697c3aefa8b01e76cc8da38e | | DARE | 40.7 | Not recorded | 697c3aefa8b01e76cc8da38e | | Base | 40.5 | Not recorded | 697c3aefa8b01e76cc8da38e | | TA | 38.5 | Not recorded | 697c3aefa8b01e76cc8da38e | | TA | 32.9 | Not recorded | 697c3aefa8b01e76cc8da38e | | TA | 28.2 | Not recorded | 697c3aefa8b01e76cc8da38e | | IP-Merging | 21.8 | Not recorded | 697c3aefa8b01e76cc8da38e | ### FRISM MathVista Acc: 79.8. Result ID: 69977dfe5698665ec4d2865a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-32B-Instruct","Reasoning Model":"QwQ-32B"}. This record does not assert independent reproduction. ### TA MathVista Acc: 78.1. Result ID: 69977dfe5698665ec4d28657. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-32B-Instruct","Reasoning Model":"QwQ-32B"}. This record does not assert independent reproduction. ### Base MathVista Acc: 77.7. Result ID: 69977dfe5698665ec4d28656. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-32B-Instruct","Reasoning Model":"QwQ-32B"}. This record does not assert independent reproduction. ### IP-Merging MathVista Acc: 75.5. Result ID: 69977dfe5698665ec4d28659. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-32B-Instruct","Reasoning Model":"QwQ-32B"}. This record does not assert independent reproduction. ### Ties-Merging MathVista Acc: 74.8. Result ID: 69977dfe5698665ec4d28658. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-32B-Instruct","Reasoning Model":"QwQ-32B"}. This record does not assert independent reproduction. ### FRISM MathVista Acc: 70.3. Result ID: 69977dfe5698665ec4d28655. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### IP-Merging MathVista Acc: 68.8. Result ID: 69977dfe5698665ec4d28653. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B","Temperature (T)":"0.3"}. This record does not assert independent reproduction. ### DARE MathVista Acc: 68.4. Result ID: 69977dfe5698665ec4d28651. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### Base MathVista Acc: 68.1. Result ID: 69977dfe5698665ec4d2864c. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### IP-Merging MathVista Acc: 67.2. Result ID: 69977dfe5698665ec4d28654. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B","Temperature (T)":"0.4"}. This record does not assert independent reproduction. ### IP-Merging MathVista Acc: 65.9. Result ID: 69977dfe5698665ec4d28652. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B","Temperature (T)":"0.2"}. This record does not assert independent reproduction. ### TA MathVista Acc: 65.4. Result ID: 69977dfe5698665ec4d2864d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B","Lambda (λ)":"0.1"}. This record does not assert independent reproduction. ### Ties-Merging MathVista Acc: 61.9. Result ID: 69977dfe5698665ec4d28650. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B"}. This record does not assert independent reproduction. ### FRISM MathVista Acc: 45. Result ID: 69977dfe5698665ec4d2864b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B"}. This record does not assert independent reproduction. ### Ties-Merging MathVista Acc: 41.8. Result ID: 69977dfe5698665ec4d28646. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B"}. This record does not assert independent reproduction. ### TA MathVista Acc: 41.5. Result ID: 69977dfe5698665ec4d28643. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B","Lambda (λ)":"0.1"}. This record does not assert independent reproduction. ### TA MathVista Acc: 41.4. Result ID: 69977dfe5698665ec4d28645. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B","Lambda (λ)":"0.2"}. This record does not assert independent reproduction. ### IP-Merging MathVista Acc: 41.1. Result ID: 69977dfe5698665ec4d28649. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B","Temperature (T)":"0.3"}. This record does not assert independent reproduction. ### IP-Merging MathVista Acc: 41.1. Result ID: 69977dfe5698665ec4d2864a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B","Temperature (T)":"0.4"}. This record does not assert independent reproduction. ### DARE MathVista Acc: 40.7. Result ID: 69977dfe5698665ec4d28647. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B"}. This record does not assert independent reproduction. ### Base MathVista Acc: 40.5. Result ID: 69977dfe5698665ec4d28642. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B"}. This record does not assert independent reproduction. ### TA MathVista Acc: 38.5. Result ID: 69977dfe5698665ec4d28644. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B","Lambda (λ)":"0.15"}. This record does not assert independent reproduction. ### TA MathVista Acc: 32.9. Result ID: 69977dfe5698665ec4d2864e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B","Lambda (λ)":"0.15"}. This record does not assert independent reproduction. ### TA MathVista Acc: 28.2. Result ID: 69977dfe5698665ec4d2864f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-7B-Instruct","Reasoning Model":"DeepSeek-R1-Distill-Qwen-7B","Lambda (λ)":"0.2"}. This record does not assert independent reproduction. ### IP-Merging MathVista Acc: 21.8. Result ID: 69977dfe5698665ec4d28648. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-VL-3B-Instruct","Reasoning Model":"SmallThinker-3B","Temperature (T)":"0.2"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.