Visual Question Answering on V*Bench
98.95AccuracyHuman
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| HumanCategory=Baseline2023.12 | 98.95 | 98.26 | 100 | |
| ConFoThinking (Qwen3-VL-8B)Backbone=Qwen3-VL-8B2026.02 | 92.1 | 94.8 | 88.1 | |
| ConFoThinking (Qwen3-VL-4B)Backbone=Qwen3-VL-4B2026.02 | 91.1 | 93.9 | 86.8 | |
| ZoomEye2026.02 | 90.6 | 93.9 | 85.5 | |
| FOCUSBase Model=Qwen-2.5-VL2025.06 | 90.58 | — | — | |
| ConFoThinking (Qwen2.5-VL-7B)Backbone=Qwen2.5-VL-7B2026.02 | 86.9 | 90.4 | 81.5 | |
| UG-SearchBase Model=LLaVA-OV-7B2025.10 | 86.9 | — | — | |
| Qwen3-VL-8B (with tools)Backbone=Qwen3-VL-8B, Thinking Tools=true2026.02 | 86.4 | 88.7 | 85.5 | |
| SpatialThinker-30BTraining Dataset=STVQA-7K, Training Protocol=RL with Dense Rewards (Ours)2025.11 | 85.9 | — | — | |
| ZoomEyeBase Model=LLaVA-OV-7B2025.10 | 85 | — | — | |
| Qwen3-VL-8BBackbone=Qwen3-VL-8B2026.02 | 84.3 | 86.1 | 81.6 | |
| Pixel-Reasoner2026.02 | 84.3 | 85.2 | 82.9 | |
| Qwen3-VL-4B (with tools)Backbone=Qwen3-VL-4B, Thinking Tools=true2026.02 | 83.2 | 85.2 | 80.3 | |
| UG-SearchBase Model=Qwen2.5-VL-7B2025.10 | 81.7 | — | — | |
| SpatialThinker-7BTraining Dataset=STVQA-7K, Training Protocol=RL with Dense Rewards (Ours)2025.11 | 81.7 | — | — | |
| DyFo2026.02 | 81.2 | 80 | 82.9 | |
| Qwen3-VL-30BModel Category=Proprietary and Open-Source MLLMs2025.11 | 81.2 | — | — | |
| Visual Sketchpad2026.02 | 80.1 | 81.7 | 77.6 | |
| InternVL3.5-8BBackbone=InternVL3.5-8B2026.02 | 79.1 | 82.6 | 73.7 | |
| Qwen-2.5-VL2025.06 | 79.06 | — | — | |
| Qwen3-VL-4BBackbone=Qwen3-VL-4B2026.02 | 78 | 80 | 75 | |
| LLaVA-OneVision-1.5-8BBackbone=LLaVA-OneVision-1.5-8B2026.02 | 78 | 79.1 | 76.3 | |
| SpatialThinker-3BTraining Dataset=STVQA-7K, Training Protocol=RL with Dense Rewards (Ours)2025.11 | 78 | — | — | |
| Qwen2.5-VL-7B + SFTTraining Dataset=STVQA-7K, Training Protocol=SFT2025.11 | 78 | — | — | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL-7B2026.02 | 76.4 | 78.2 | 73.6 | |
| Qwen2.5-VL-7BModel Category=Proprietary and Open-Source MLLMs2025.11 | 75.9 | — | — | |
| SEALLLM Backbone=Vicuna-7B, Category=Ours2023.12 | 75.39 | 74.78 | 76.31 | |
| SEAL2026.02 | 74.9 | 78.3 | 68.7 | |
| Qwen2.5-VL-3BModel Category=Proprietary and Open-Source MLLMs2025.11 | 74.9 | — | — | |
| LLaVA-OV-7BBase Model=LLaVA-OV-7B2025.10 | 74.4 | — | — | |
| Qwen2.5-VL-3B + Vanilla GRPOTraining Dataset=STVQA-7K, Training Protocol=Vanilla GRPO2025.11 | 74.3 | — | — | |
| Qwen2.5-VL-7B + Vanilla GRPOTraining Dataset=STVQA-7K, Training Protocol=Vanilla GRPO2025.11 | 73.9 | — | — | |
| GPT-5-0807Model Category=Proprietary and Open-Source MLLMs2025.11 | 73.3 | — | — | |
| Qwen2.5-VL-3B + SFTTraining Dataset=STVQA-7K, Training Protocol=SFT2025.11 | 73.3 | — | — | |
| ViCropBase Model=Qwen2.5-VL-7B2025.10 | 69.6 | — | — | |
| ThymeBase Model=Qwen2.5-VL-7B2025.10 | 68.1 | — | — | |
| TextCoTBase Model=Qwen2.5-VL-7B2025.10 | 67 | — | — | |
| GPT-4o-0513Model Category=Proprietary and Open-Source MLLMs2025.11 | 66 | — | — | |
| Qwen2.5-VL-7BBase Model=Qwen2.5-VL-7B2025.10 | 64.9 | — | — | |
| ViCrop2026.02 | 62.3 | 65.2 | 57.9 | |
| ICoT2026.02 | 60.7 | 63.5 | 56.6 | |
| Claude-4-Sonnet-0514Model Category=Proprietary and Open-Source MLLMs2025.11 | 60.7 | — | — | |
| VLAA-Thinker-7BModel Category=Proprietary and Open-Source MLLMs2025.11 | 58.1 | — | — | |
| SpaceThinkerModel Category=Proprietary and Open-Source MLLMs2025.11 | 56.5 | — | — | |
| SpaceOmModel Category=Proprietary and Open-Source MLLMs2025.11 | 56.5 | — | — | |
| GPT-4VCategory=Commercial chatbot systems2023.12 | 54.97 | 51.3 | 60.52 | |
| Claude-3.5-Sonnet-0620Model Category=Proprietary and Open-Source MLLMs2025.11 | 51.8 | — | — | |
| LLaVA-1.5Category=Open-source end-to-end MLLMs2023.12 | 48.68 | 43.47 | 56.57 | |
| Gemini ProCategory=Commercial chatbot systems2023.12 | 48.16 | 40.86 | 59.21 | |
| Ours (PSP & VRG)Model=LaViDa, Generation length L / step T=128/642026.04 | 46.6 | — | — | |
| Ours (PSP)Model=LaViDa, Generation length L / step T=256/1282026.04 | 46 | — | — | |
| Ours (PSP & VRG)Model=LaViDa, Generation length L / step T=256/1282026.04 | 46 | — | — | |
| Ours (PSP)Model=LaViDa, Generation length L / step T=128/642026.04 | 45.5 | — | — | |
| Ours (PSP & VRG)Model=LaViDa, Generation length L / step T=64/322026.04 | 45.5 | — | — | |
| MarginModel=LaViDa, Generation length L / step T=256/1282026.04 | 44.5 | — | — | |
| Low-confModel=LaViDa, Generation length L / step T=256/1282026.04 | 44.5 | — | — | |
| Ours (PSP)Model=LaViDa, Generation length L / step T=64/322026.04 | 44.5 | — | — | |
| DDCoTModel=LaViDa, Generation length L / step T=256/1282026.04 | 43.9 | — | — | |
| CCoTModel=LaViDa, Generation length L / step T=128/642026.04 | 43.9 | — | — | |
| MarginModel=LaViDa, Generation length L / step T=64/322026.04 | 43.4 | — | — | |
| MarginModel=LaViDa, Generation length L / step T=128/642026.04 | 43.4 | — | — | |
| Low-confModel=LaViDa, Generation length L / step T=128/642026.04 | 43.4 | — | — | |
| CCoTModel=LaViDa, Generation length L / step T=64/322026.04 | 42.9 | — | — | |
| EntropyModel=LaViDa, Generation length L / step T=256/1282026.04 | 42.9 | — | — | |
| Low-confModel=LaViDa, Generation length L / step T=64/322026.04 | 42.9 | — | — | |
| DDCoTModel=LaViDa, Generation length L / step T=128/642026.04 | 42.4 | — | — | |
| CCoTModel=LaViDa, Generation length L / step T=256/1282026.04 | 42.4 | — | — | |
| EntropyModel=LaViDa, Generation length L / step T=64/322026.04 | 42.4 | — | — | |
| EntropyModel=LaViDa, Generation length L / step T=128/642026.04 | 41.8 | — | — | |
| MM-ReactCategory=LLM tool-using pipelines2023.12 | 41.36 | 34.78 | 51.31 | |
| VisprogCategory=LLM tool-using pipelines2023.12 | 41.36 | 31.3 | 56.57 | |
| DDCoTModel=LaViDa, Generation length L / step T=64/322026.04 | 41.3 | — | — | |
| OtterCategory=Open-source end-to-end MLLMs2023.12 | 38.74 | 26.95 | 56.57 | |
| Ours (PSP & VRG)Model=MMaDa, Generation length L / step T=128/642026.04 | 38.7 | — | — | |
| MiniGPT-4Category=Open-source end-to-end MLLMs2023.12 | 38.22 | 30.43 | 50 | |
| Ours (PSP)Model=MMaDa, Generation length L / step T=128/642026.04 | 38.2 | — | — | |
| Ours (PSP & VRG)Model=MMaDa, Generation length L / step T=64/322026.04 | 38.2 | — | — | |
| Ours (PSP)Model=MMaDa, Generation length L / step T=64/322026.04 | 37.7 | — | — | |
| Ours (PSP & VRG)Model=MMaDa, Generation length L / step T=256/1282026.04 | 37.7 | — | — | |
| BLIP2Category=Open-source end-to-end MLLMs2023.12 | 37.69 | 26.95 | 53.94 | |
| VisualChatGPTCategory=LLM tool-using pipelines2023.12 | 37.69 | 30.43 | 48.68 | |
| BardCategory=Commercial chatbot systems2023.12 | 37.17 | 31.3 | 46.05 | |
| Ours (PSP)Model=MMaDa, Generation length L / step T=256/1282026.04 | 37.1 | — | — | |
| CCoTModel=MMaDa, Generation length L / step T=256/1282026.04 | 36.6 | — | — | |
| DDCoTModel=MMaDa, Generation length L / step T=256/1282026.04 | 36.1 | — | — | |
| CCoTModel=MMaDa, Generation length L / step T=64/322026.04 | 36.1 | — | — | |
| CCoTModel=MMaDa, Generation length L / step T=128/642026.04 | 36.1 | — | — | |
| EntropyModel=MMaDa, Generation length L / step T=64/322026.04 | 36.1 | — | — | |
| Random GuessCategory=Baseline2023.12 | 35.99 | 26.73 | 50 | |
| EntropyModel=MMaDa, Generation length L / step T=128/642026.04 | 35.6 | — | — | |
| MarginModel=MMaDa, Generation length L / step T=128/642026.04 | 35.6 | — | — | |
| Low-confModel=MMaDa, Generation length L / step T=64/322026.04 | 35.6 | — | — | |
| LLaVACategory=Open-source end-to-end MLLMs2023.12 | 35.59 | 23.47 | 53.94 | |
| DDCoTModel=MMaDa, Generation length L / step T=64/322026.04 | 35 | — | — | |
| EntropyModel=MMaDa, Generation length L / step T=256/1282026.04 | 35 | — | — | |
| Low-confModel=MMaDa, Generation length L / step T=128/642026.04 | 35 | — | — | |
| DDCoTModel=MMaDa, Generation length L / step T=128/642026.04 | 34.5 | — | — | |
| MarginModel=MMaDa, Generation length L / step T=256/1282026.04 | 34.5 | — | — | |
| Low-confModel=MMaDa, Generation length L / step T=256/1282026.04 | 34.5 | — | — | |
| InstructBLIPCategory=Open-source end-to-end MLLMs2023.12 | 34.02 | 25.21 | 47.36 |