Multimodal Mathematical Reasoning on Math-V
68.8Accuracy (ALL)Human
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HumanCategory=Heuristics Baselines2025.07 | 68.8 | 55.1 | 78.6 | 99.6 | 98.4 | 43.5 | 98.5 | 91.3 | 62.2 | 61.3 | 33.5 | 47.2 | 73.5 | 87.3 | 93.1 | 99.8 | 69 | |
| HumanType=Heuristic Baselines2025.07 | 68.8 | 55.1 | 78.6 | 99.6 | 98.4 | 43.5 | 98.5 | 91.3 | 62.2 | 61.3 | 33.5 | 47.2 | 73.5 | 87.3 | 93.1 | 99.8 | 69 | |
| GPT-4oCategory=Close-Source MLLMs2025.07 | 30.4 | 42 | 39.3 | 49.3 | 28.9 | 25.6 | 22.4 | 24 | 23.3 | 29.4 | 17.3 | 29.8 | 30.1 | 29.1 | 44.8 | 34.8 | 17.9 | |
| GPT-4oModel Category=Close-Source MLLM2025.07 | 30.4 | 42 | 39.3 | 49.3 | 28.9 | 25.6 | 22.4 | 24 | 23.3 | 29.4 | 17.3 | 29.8 | 30.1 | 29.1 | 44.8 | 34.8 | 17.9 | |
| Qwen2.5-VL-DPCategory=Open-Source MLLMs, Training=DP2025.07 | 26.9 | 23.3 | 30.8 | 32.2 | 20.6 | 27.3 | 17.4 | 23.9 | 22.9 | 28.6 | 28.9 | 30.9 | 28.8 | 28.7 | 37.9 | 18.4 | 23.2 | |
| Qwen2.5-VL-DPModel Category=Open-Source MLLM, Reinforcement Learning=DP2025.07 | 26.9 | 23.3 | 30.8 | 32.2 | 20.6 | 27.3 | 17.4 | 23.9 | 22.9 | 28.6 | 28.9 | 30.9 | 28.8 | 28.7 | 37.9 | 18.4 | 23.2 | |
| Qwen2.5-VL-7BCategory=Open-Source MLLMs, Parameters=7B2025.07 | 25 | 22 | 29.8 | 32.1 | 19.5 | 18.5 | 16.4 | 22.1 | 11.1 | 25.2 | 29.3 | 27.6 | 28.5 | 22.9 | 34.5 | 17.4 | 22 | |
| Qwen2.5-VL-7BModel Category=Open-Source MLLM, Parameters=7B2025.07 | 25 | 22 | 29.8 | 32.1 | 19.5 | 18.5 | 16.4 | 22.1 | 11.1 | 25.2 | 29.3 | 27.6 | 28.5 | 22.9 | 34.5 | 17.4 | 22 | |
| GPT-4VCategory=Close-Source MLLMs2025.07 | 22.8 | 27.3 | 32.1 | 35.7 | 21.1 | 16.7 | 13.4 | 22.1 | 14.4 | 16.8 | 22 | 22.2 | 20.9 | 23.8 | 24.1 | 21.7 | 25.6 | |
| GPT-4VModel Category=Close-Source MLLM2025.07 | 22.8 | 27.3 | 32.1 | 35.7 | 21.1 | 16.7 | 13.4 | 22.1 | 14.4 | 16.8 | 22 | 22.2 | 20.9 | 23.8 | 24.1 | 21.7 | 25.6 | |
| Gemini ProCategory=Close-Source MLLMs2025.07 | 17.7 | 15.1 | 10.7 | 20.7 | 20.1 | 11.9 | 7.5 | 20.2 | 21.1 | 16.8 | 19.1 | 19 | 20 | 14.3 | 13.8 | 17.4 | 20.8 | |
| Qwen2-VL-DPCategory=Open-Source MLLMs, Training=DP2025.07 | 17.7 | 15.2 | 20.8 | 20.8 | 20.2 | 12 | 7.9 | 20.3 | 21.2 | 16.9 | 19.2 | 19.1 | 23.2 | 14.4 | 13.9 | 17.5 | 20.9 | |
| Gemini ProModel Category=Close-Source MLLM2025.07 | 17.7 | 15.1 | 10.7 | 20.7 | 20.1 | 11.9 | 7.5 | 20.2 | 21.1 | 16.8 | 19.1 | 19 | 20 | 14.3 | 13.8 | 17.4 | 20.8 | |
| Qwen2-VL-DPModel Category=Open-Source MLLM, Reinforcement Learning=DP2025.07 | 17.7 | 15.2 | 20.8 | 20.8 | 20.2 | 12 | 7.9 | 20.3 | 21.2 | 16.9 | 19.2 | 19.1 | 23.2 | 14.4 | 13.9 | 17.5 | 20.9 | |
| Qwen2-VL-7BCategory=Open-Source MLLMs, Parameters=7B2025.07 | 16.3 | 11.3 | 24.9 | 15.7 | 16.9 | 10.1 | 11.9 | 16.4 | 15.6 | 19.3 | 22.5 | 16.4 | 22.5 | 14.3 | 17.2 | 4.4 | 20.8 | |
| Qwen2-VL-7BModel Category=Open-Source MLLM, Parameters=7B2025.07 | 16.3 | 11.3 | 24.9 | 15.7 | 16.9 | 10.1 | 11.9 | 16.4 | 15.6 | 19.3 | 22.5 | 16.4 | 22.5 | 14.3 | 17.2 | 4.4 | 20.8 | |
| Math-LLaVACategory=Open-Source MLLMs2025.07 | 15.7 | 9 | 20.2 | 15.7 | 18.2 | 10.1 | 10.5 | 16.4 | 14.4 | 16 | 20.2 | 18.4 | 17.6 | 9.4 | 24.1 | 21.7 | 17.9 | |
| Math-LLaVAModel Category=Open-Source MLLM2025.07 | 15.7 | 9 | 20.2 | 15.7 | 18.2 | 10.1 | 10.5 | 16.4 | 14.4 | 16 | 20.2 | 18.4 | 17.6 | 9.4 | 24.1 | 21.7 | 17.9 | |
| Qwen-VL-MaxCategory=Close-Source MLLMs2025.07 | 15.6 | 10.7 | 19.1 | 20 | 16.9 | 12.5 | 17.9 | 16.4 | 12.2 | 21 | 13.3 | 14.2 | 19.8 | 11.5 | 20.7 | 13 | 17.3 | |
| Qwen-VL-MaxModel Category=Close-Source MLLM2025.07 | 15.6 | 10.7 | 19.1 | 20 | 16.9 | 12.5 | 17.9 | 16.4 | 12.2 | 21 | 13.3 | 14.2 | 19.8 | 11.5 | 20.7 | 13 | 17.3 | |
| LLaVA-1.5-13BCategory=Open-Source MLLMs, Parameters=13B2025.07 | 11.1 | 7 | 14.3 | 14.3 | 9.1 | 6.6 | 6 | 13.5 | 5.6 | 13.5 | 10.4 | 12.6 | 14.7 | 11.5 | 13.8 | 13 | 10.7 | |
| LLaVA-1.5-13BModel Category=Open-Source MLLM, Parameters=13B2025.07 | 11.1 | 7 | 14.3 | 14.3 | 9.1 | 6.6 | 6 | 13.5 | 5.6 | 13.5 | 10.4 | 12.6 | 14.7 | 11.5 | 13.8 | 13 | 10.7 | |
| Qwen-VL-PlusCategory=Close-Source MLLMs2025.07 | 10.7 | 11.3 | 17.9 | 14.3 | 12.7 | 4.8 | 10.5 | 15.4 | 8.9 | 14.3 | 11.6 | 6.4 | 10 | 14.3 | 6.9 | 8.7 | 11.3 | |
| Qwen-VL-PlusModel Category=Close-Source MLLM2025.07 | 10.7 | 11.3 | 17.9 | 14.3 | 12.7 | 4.8 | 10.5 | 15.4 | 8.9 | 14.3 | 11.6 | 6.4 | 10 | 14.3 | 6.9 | 8.7 | 11.3 | |
| SPHINX-V2-13BCategory=Open-Source MLLMs, Parameters=13B2025.07 | 9.7 | 6.7 | 7.1 | 12.9 | 7.5 | 7.7 | 6 | 9.6 | 16.7 | 10.1 | 11 | 11.8 | 12.5 | 8.2 | 8.6 | 8.7 | 6 | |
| SPHINX-V2-13BModel Category=Open-Source MLLM, Parameters=13B2025.07 | 9.7 | 6.7 | 7.1 | 12.9 | 7.5 | 7.7 | 6 | 9.6 | 16.7 | 10.1 | 11 | 11.8 | 12.5 | 8.2 | 8.6 | 8.7 | 6 | |
| Random ChanceCategory=Heuristics Baselines2025.07 | 7.2 | 1.5 | 11.9 | 7.1 | 9.7 | 4.8 | 6 | 22.1 | 1.1 | 7.6 | 0.6 | 9.4 | 6.7 | 8.2 | 8.6 | 13 | 7.1 | |
| Random ChanceType=Heuristic Baselines2025.07 | 7.2 | 1.5 | 11.9 | 7.1 | 9.7 | 4.8 | 6 | 22.1 | 1.1 | 7.6 | 0.6 | 9.4 | 6.7 | 8.2 | 8.6 | 13 | 7.1 |