Spatial Reasoning on RealWorldQA
69.67AccuracyFull Model
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Full ModelBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=0%2026.02 | 69.67 | — | — | |
| VAPO-ThinkerModel=VAPO-Thinker2025.09 | 69.4 | 73.7 | — | |
| POPBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=33.3%2026.02 | 69.28 | — | — | |
| Qwen2.5-VLScale=32B2026.05 | 68.1 | — | — | |
| WandaBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=30%2026.02 | 67.45 | — | — | |
| Vision-R1Model=Vision-R12025.09 | 66.8 | 70.5 | — | |
| Qwen2.5-VL + DRScaffoldScale=3B2026.05 | 66.8 | — | — | |
| VLAA-ThinkerModel=VLAA-Thinker2025.09 | 66.3 | 70.6 | — | |
| o4-miniModel Category=Proprietary, Tool Use=true2025.12 | 65.9 | — | — | |
| R1-OneVisionModel=R1-OneVision2025.09 | 65.7 | 69.6 | — | |
| Qwen2.5-VLScale=3B2026.05 | 65.4 | — | — | |
| Phi4-multimodal + DRScaffoldScale=5.6B2026.05 | 65.4 | — | — | |
| Phi4-multimodalScale=5.6B2026.05 | 64.4 | — | — | |
| Qwen2.5-VLModel=Qwen2.5-VL2025.09 | 64.3 | 67.9 | — | |
| Qwen2-VL#Params.=2B2026.03 | 62.9 | — | — | |
| Gemini-2.5-FlashModel Category=Proprietary, Tool Use=true2025.12 | 62.2 | — | — | |
| CHEERS#Params.=1.5B2026.03 | 60.9 | — | — | |
| InternVL2.5 + DRScaffoldScale=2B2026.05 | 60.9 | — | — | |
| InternVL2.5Scale=2B2026.05 | 60.6 | — | — | |
| Emu3#Params.=8B2026.03 | 57.4 | — | — | |
| VALORModel Category=Open-Source, Tool Use=true2025.12 | 57.3 | — | — | |
| Show-o2#Params.=1.5B2026.03 | 56.5 | — | — | |
| SiTebackbone=LLaVA-flickr, variant=ratio2025.12 | 56.25 | — | — | |
| POPBackbone=Gemma-3-12B-It, Pruning Ratio=33.3%2026.02 | 55.42 | — | — | |
| WandaBackbone=Gemma-3-12B-It, Pruning Ratio=30%2026.02 | 55.29 | — | — | |
| SiTebackbone=LLaVA-flickr, variant=rand2025.12 | 55.28 | — | — | |
| SiTebackbone=LLaVA, variant=ratio2025.12 | 55.26 | — | — | |
| SiTebackbone=LLaVA, variant=rand2025.12 | 55.21 | — | — | |
| LLaVAvariant=Baseline2025.12 | 55.12 | — | — | |
| Full ModelBackbone=Gemma-3-12B-It, Pruning Ratio=0%2026.02 | 54.64 | — | — | |
| LLaVAvariant=flickr2025.12 | 54.51 | — | — | |
| GPT-4oModel Category=Proprietary, Tool Use=true2025.12 | 54.5 | — | — | |
| SiTebackbone=Qwen2, variant=rand2025.12 | 54.44 | — | — | |
| Claude-3.5-HaikuModel Category=Proprietary, Tool Use=true2025.12 | 54.4 | — | — | |
| VALOR-RLModel Category=Open-Source, Tool Use=true2025.12 | 53.5 | — | — | |
| Qwen3-8BModel Category=Open-Source, Tool Use=true2025.12 | 53.3 | — | — | |
| ShortGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.02 | 53.07 | — | — | |
| SiTebackbone=Qwen2, variant=ratio2025.12 | 52.94 | — | — | |
| Janus-Pro#Params.=1.5B2026.03 | 52.6 | — | — | |
| Qwen2variant=Baseline2025.12 | 52.28 | — | — | |
| mPLUG-Owl2#Params.=7B2026.03 | 50.3 | — | — | |
| Harmon#Params.=1.5B2026.03 | 49.8 | — | — | |
| SiTebackbone=Qwen2-flickr, variant=ratio2025.12 | 49.74 | — | — | |
| Gemini-2.0-FlashModel Category=Proprietary, Tool Use=true2025.12 | 47.3 | — | — | |
| Gemma-3-12BModel Category=Open-Source, Tool Use=true2025.12 | 47.3 | — | — | |
| Llama-3.2-11BModel Category=Open-Source, Tool Use=true2025.12 | 46.9 | — | — | |
| Qwen2variant=flickr2025.12 | 43.2 | — | — | |
| SiTebackbone=Qwen2-flickr, variant=rand2025.12 | 42.75 | — | — | |
| JanusFlow#Params.=1.3B2026.03 | 41.2 | — | — | |
| SliceGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.02 | 32.55 | — | — | |
| SliceGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.02 | 5.23 | — | — | |
| ShortGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.02 | 0.39 | — | — | |
| CoT promptingBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=CoT prompting2026.04 | — | — | 63.05 | |
| CoT promptingBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=CoT prompting2026.04 | — | — | 62.35 | |
| FGRPOBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=Faithful GRPO2026.04 | — | — | 67.64 | |
| FGRPOBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=Faithful GRPO2026.04 | — | — | 60.7 | |
| GPT-4o (CoT)Reasoning Strategy=CoT2026.04 | — | — | 73.59 | |
| GPT-5-nano (CoT)Reasoning Strategy=CoT2026.04 | — | — | 71.9 | |
| GRPO-TBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=RL (Task Reward Only)2026.04 | — | — | 66.67 | |
| GRPO-TBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=RL (Task Reward Only)2026.04 | — | — | 61.83 | |
| Non-reasoningBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=Non-reasoning2026.04 | — | — | 69.02 | |
| Non-reasoningBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=Non-reasoning2026.04 | — | — | 65.88 | |
| R1-OneVisionReasoning Strategy=MRM baseline2026.04 | — | — | 49.87 | |
| TreeVGRReasoning Strategy=MRM baseline2026.04 | — | — | 66.8 | |
| ViGoRL-SpatialReasoning Strategy=MRM baseline2026.04 | — | — | 65.67 | |
| Vision-R1Reasoning Strategy=MRM baseline2026.04 | — | — | 67.41 | |
| VL-RethinkerReasoning Strategy=MRM baseline2026.04 | — | — | 68.5 |