Mathematical Visual Question Answering on MATH-Vision Full (test)
73.3Relaxed AccuracyGemini 2.5 Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini 2.5 ProModel Category=Proprietary VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 73.3 | |
| HumanModel Category=Baseline, Evaluation Source=Model authors or leaderboards2025.09 | 68.82 | |
| Qwen2.5-VL-7B-Instruct + Visual-TableQAModel Category=Finetuned VLMs, Evaluation Source=LLM jury2025.09 | 49.77 | |
| Qwen2.5-VL-7B-Instruct + ReachQAModel Category=Finetuned VLMs, Evaluation Source=LLM jury2025.09 | 48.57 | |
| Llama 4 Maverick 17B-128E InstructModel Category=Open-Source VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 45.89 | |
| Gemini 2.5 FlashModel Category=Proprietary VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 41.3 | |
| Qwen2.5-VL-32B-InstructModel Category=Open-Source VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 38.1 | |
| Claude 3.5 SonnetModel Category=Proprietary VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 32.76 | |
| Mistral Small 3.1 24B InstructModel Category=Open-Source VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 32.45 | |
| GPT-4oModel Category=Proprietary VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 30.39 | |
| GPT-4o miniModel Category=Proprietary VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 28.85 | |
| Qwen2.5-VL-7B-InstructModel Category=Open-Source VLMs, Evaluation Source=Model authors or leaderboards2025.09 | 25.1 |