List your company
ResearchFormerly WizWand

Spatial Reasoning on RealWorldQA

69.67AccuracyFull Model
-2.381216.324435.0353.7356Sep 30, 2025Nov 8, 2025Dec 18, 2025Jan 26, 2026Mar 7, 2026Apr 15, 2026May 25, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Full ModelBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=0%2026.0269.67——
VAPO-ThinkerModel=VAPO-Thinker2025.0969.473.7—
POPBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=33.3%2026.0269.28——
Qwen2.5-VLScale=32B2026.0568.1——
WandaBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=30%2026.0267.45——
Vision-R1Model=Vision-R12025.0966.870.5—
Qwen2.5-VL + DRScaffoldScale=3B2026.0566.8——
VLAA-ThinkerModel=VLAA-Thinker2025.0966.370.6—
o4-miniModel Category=Proprietary, Tool Use=true2025.1265.9——
R1-OneVisionModel=R1-OneVision2025.0965.769.6—
Qwen2.5-VLScale=3B2026.0565.4——
Phi4-multimodal + DRScaffoldScale=5.6B2026.0565.4——
Phi4-multimodalScale=5.6B2026.0564.4——
Qwen2.5-VLModel=Qwen2.5-VL2025.0964.367.9—
Qwen2-VL#Params.=2B2026.0362.9——
Gemini-2.5-FlashModel Category=Proprietary, Tool Use=true2025.1262.2——
CHEERS#Params.=1.5B2026.0360.9——
InternVL2.5 + DRScaffoldScale=2B2026.0560.9——
InternVL2.5Scale=2B2026.0560.6——
Emu3#Params.=8B2026.0357.4——
VALORModel Category=Open-Source, Tool Use=true2025.1257.3——
Show-o2#Params.=1.5B2026.0356.5——
SiTebackbone=LLaVA-flickr, variant=ratio2025.1256.25——
POPBackbone=Gemma-3-12B-It, Pruning Ratio=33.3%2026.0255.42——
WandaBackbone=Gemma-3-12B-It, Pruning Ratio=30%2026.0255.29——
SiTebackbone=LLaVA-flickr, variant=rand2025.1255.28——
SiTebackbone=LLaVA, variant=ratio2025.1255.26——
SiTebackbone=LLaVA, variant=rand2025.1255.21——
LLaVAvariant=Baseline2025.1255.12——
Full ModelBackbone=Gemma-3-12B-It, Pruning Ratio=0%2026.0254.64——
LLaVAvariant=flickr2025.1254.51——
GPT-4oModel Category=Proprietary, Tool Use=true2025.1254.5——
SiTebackbone=Qwen2, variant=rand2025.1254.44——
Claude-3.5-HaikuModel Category=Proprietary, Tool Use=true2025.1254.4——
VALOR-RLModel Category=Open-Source, Tool Use=true2025.1253.5——
Qwen3-8BModel Category=Open-Source, Tool Use=true2025.1253.3——
ShortGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.0253.07——
SiTebackbone=Qwen2, variant=ratio2025.1252.94——
Janus-Pro#Params.=1.5B2026.0352.6——
Qwen2variant=Baseline2025.1252.28——
mPLUG-Owl2#Params.=7B2026.0350.3——
Harmon#Params.=1.5B2026.0349.8——
SiTebackbone=Qwen2-flickr, variant=ratio2025.1249.74——
Gemini-2.0-FlashModel Category=Proprietary, Tool Use=true2025.1247.3——
Gemma-3-12BModel Category=Open-Source, Tool Use=true2025.1247.3——
Llama-3.2-11BModel Category=Open-Source, Tool Use=true2025.1246.9——
Qwen2variant=flickr2025.1243.2——
SiTebackbone=Qwen2-flickr, variant=rand2025.1242.75——
JanusFlow#Params.=1.3B2026.0341.2——
SliceGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.0232.55——
SliceGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.025.23——
ShortGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.020.39——
CoT promptingBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=CoT prompting2026.04——63.05
CoT promptingBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=CoT prompting2026.04——62.35
FGRPOBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=Faithful GRPO2026.04——67.64
FGRPOBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=Faithful GRPO2026.04——60.7
GPT-4o (CoT)Reasoning Strategy=CoT2026.04——73.59
GPT-5-nano (CoT)Reasoning Strategy=CoT2026.04——71.9
GRPO-TBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=RL (Task Reward Only)2026.04——66.67
GRPO-TBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=RL (Task Reward Only)2026.04——61.83
Non-reasoningBackbone=Qwen2.5-VL-7B-Instruct, Model Scale=7B, Reasoning Strategy=Non-reasoning2026.04——69.02
Non-reasoningBackbone=Qwen2.5-VL-3B-Instruct, Model Scale=3B, Reasoning Strategy=Non-reasoning2026.04——65.88
R1-OneVisionReasoning Strategy=MRM baseline2026.04——49.87
TreeVGRReasoning Strategy=MRM baseline2026.04——66.8
ViGoRL-SpatialReasoning Strategy=MRM baseline2026.04——65.67
Vision-R1Reasoning Strategy=MRM baseline2026.04——67.41
VL-RethinkerReasoning Strategy=MRM baseline2026.04——68.5