Mathematical Reasoning on MathVista (test)
84.44AccuracyETTC
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ETTCAggregation Method=ETTC, Ensemble=Qwen family2026.05 | 84.44 | — | — | — | — | — | |
| VotingAggregation Method=Voting, Ensemble=Qwen family2026.05 | 83.15 | — | — | — | — | — | |
| Qwen-72BModel Scale=72B, Family=Qwen2026.05 | 80.58 | — | — | — | — | — | |
| Qwen-32BModel Scale=32B, Family=Qwen2026.05 | 78.58 | — | — | — | — | — | |
| Qwen-7BModel Scale=7B, Family=Qwen2026.05 | 72.08 | — | — | — | — | — | |
| AverageAggregation Method=Mean, Ensemble=Qwen family2026.05 | 70.8 | — | — | — | — | — | |
| Qwen2.5-VL-7B + Cont. Rewardreward_type=Continuous2025.11 | 70.52 | — | — | — | — | — | |
| Qwen2.5-VL-7B + Discrete Rewardreward_type=Discrete2025.11 | 68.88 | — | — | — | — | — | |
| Qwen2.5-VL-7B (Baseline)2025.11 | 68.46 | — | — | — | — | — | |
| Vision-Zeroexternal supervision=true2025.11 | 68.43 | — | — | — | — | — | |
| MM-Verifier + MM-ReasonerSource=Proposed Method2025.02 | 65.3 | 60.1 | 73.1 | 68.8 | 67.7 | 55.9 | |
| GPT-4oSource=Closed-Source MLLM2025.02 | 63.8 | 64.7 | — | — | — | — | |
| HumanSource=Baseline2025.02 | 60.3 | 48.4 | 73 | 59.7 | 63.2 | 55.9 | |
| MMCTAgentsetup=zero-shot, critic=true2024.05 | 56.5 | — | — | — | — | — | |
| CNTPModel=LLaVA-CoT2025.07 | 55.7 | — | — | — | — | — | |
| Stochastic DecodingModel=LLaVA-CoT2025.07 | 55.3 | — | — | — | — | — | |
| Stochastic DecodingModel=Llama-3.2-11B-Vision-Instruct2025.07 | 53.4 | — | — | — | — | — | |
| MMCTAgentsetup=zero-shot, critic=false2024.05 | 53.3 | — | — | — | — | — | |
| Gemini 1.0 Ultrasetup=zero-shot2024.05 | 53 | — | — | — | — | — | |
| Qwen2-VL-7BSource=Open-Source MLLM2025.02 | 52.5 | 44.7 | 59.7 | 55.4 | 52.5 | 49.7 | |
| Gemini 1.5 Prosetup=zero-shot2024.05 | 52.1 | — | — | — | — | — | |
| Qwen-3BModel Scale=3B, Family=Qwen2026.05 | 51.94 | — | — | — | — | — | |
| LLaVA-OneVision-7BSource=Open-Source MLLM2025.02 | 51.1 | 66.8 | 54.8 | 39.4 | 57.6 | 40.8 | |
| Claude 3 Opussetup=zero-shot2024.05 | 50.5 | — | — | — | — | — | |
| GPT-4Vsetup=zero-shot2024.05 | 49.9 | — | — | — | — | — | |
| CNTPModel=Llama-3.2-11B-Vision-Instruct2025.07 | 49.2 | — | — | — | — | — | |
| Greedy DecodingModel=LLaVA-CoT2025.07 | 48.8 | — | — | — | — | — | |
| Claude 3 Sonnetsetup=zero-shot2024.05 | 47.9 | — | — | — | — | — | |
| Greedy DecodingModel=Llama-3.2-11B-Vision-Instruct2025.07 | 47.8 | — | — | — | — | — | |
| Math-LLaVA-13BSource=Open-Source MLLM2025.02 | 46.6 | 57.7 | 56.5 | 37.2 | 51.3 | 33.5 | |
| llama-3.2-11B-VisionSource=Open-Source MLLM2025.02 | 46.5 | 38 | 51.6 | 50.9 | 51.9 | 39.7 | |
| Claude 3 Haikusetup=zero-shot2024.05 | 46.4 | — | — | — | — | — | |
| Gemini 1.0 Prosetup=zero-shot2024.05 | 45.2 | — | — | — | — | — | |
| Qwen-VL-PlusSource=Closed-Source MLLM2025.02 | 43.3 | 35.5 | 31.2 | 54.6 | 48.1 | 51.4 | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Yi-34B2024.05 | 40 | — | — | — | — | — | |
| Mini-GeminiPT+IT=1.2M+1.5M, Res.=336+768, LLM=Vicuna-13B2024.05 | 37 | — | — | — | — | — | |
| SPHINX-PlusPT+IT=16M, Res.=448, LLM=Llama2-13B2024.05 | 36.8 | — | — | — | — | — | |
| SPHINX-V2-13BSource=Open-Source MLLM2025.02 | 36.7 | 16.4 | 23.1 | 54.6 | 41.8 | 43 | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384, LLM=Vicuna-13B2024.05 | 36.5 | — | — | — | — | — | |
| MM1PT+IT=3B+1.4M, Res.=1344, LLM=MM1-7B2024.05 | 35.9 | — | — | — | — | — | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Vicuna-13B2024.05 | 35.5 | — | — | — | — | — | |
| LLaVA-NeXTPT+IT=0.5M+0.7M, Res.=336 AnyRes, LLM=Vicuna-13B2024.05 | 35.3 | — | — | — | — | — | |
| Deepseek-VLSource=Open-Source MLLM2025.02 | 34.9 | 28.4 | 15.2 | 26.8 | 32.9 | 34.6 | |
| CogVLM-ChatLLM=Vicuna-7B2023.11 | 34.5 | — | — | — | — | — | |
| Qwen-VL-ChatLLM=Qwen-7B2023.11 | 33.8 | — | — | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Yi-34B LORA2024.05 | 33.5 | — | — | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Llama3-70B LORA2024.05 | 32.9 | — | — | — | — | — | |
| Dense ConnectorPT+IT=1.2M+1.5M, Res.=384 AnyRes, LLM=Vicuna-7B2024.05 | 32.7 | — | — | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Vicuna-13B2024.05 | 29.6 | — | — | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Llama3-8B2024.05 | 28.6 | — | — | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Phi2-2.7B2024.05 | 28.2 | — | — | — | — | — | |
| SPHINX-2kLLM=LLaMA2 13B2023.11 | 27.8 | — | — | — | — | — | |
| LLaVA-1.5-13BSource=Open-Source MLLM2025.02 | 27.7 | 22.7 | 18.9 | 23.8 | 43 | 30.2 | |
| LLaVA-v1.5PT+IT=0.5M+0.6M, Res.=336, LLM=Vicuna-13B2024.05 | 27.6 | — | — | — | — | — | |
| LLaVA-1.5LLM=Vicuna-13B2023.11 | 26.1 | — | — | — | — | — | |
| Dense ConnectorPT+IT=0.5M+0.6M, Res.=384, LLM=Vicuna-7B2024.05 | 25.5 | — | — | — | — | — | |
| InstructBLIPLLM=Vicuna-7B2023.11 | 25.3 | — | — | — | — | — | |
| mPLUG-Owl2LLM=LLaMA2-7B2023.11 | 25.3 | — | — | — | — | — | |
| G-LLaVA-7BSource=Open-Source MLLM2025.02 | 25.1 | 48.7 | 3.6 | 19.1 | 25 | 28.7 | |
| LLaVA-1.5LLM=Vicuna-7B2023.11 | 23.6 | — | — | — | — | — | |
| MiniGPT-4LLM=Vicuna-7B2023.11 | 23.1 | — | — | — | — | — | |
| MiniGPT4-7BSource=Open-Source MLLM2025.02 | 23.1 | 26 | 13.4 | 18.6 | 30.4 | 30.2 | |
| mPLUG-Owl2PT+IT=348M+1.2M, Res.=448, LLM=Llama2-7B2024.05 | 22.2 | — | — | — | — | — | |
| mPLUG-Owl2-7BSource=Open-Source MLLM2025.02 | 22.2 | 23.6 | 10.2 | 22.7 | 27.2 | 27.9 | |
| OpenFlamingoLLM=MPT-7B2023.11 | 18.6 | — | — | — | — | — | |
| RandomSource=Baseline2025.02 | 17.9 | 21.6 | 3.8 | 18.2 | 19.6 | 26.3 |