Spatial Reasoning on SITE
67.5AccuracyHuman
Evaluation results
| Method | Links | |
|---|---|---|
| Human2026.02 | 67.5 | |
| GEMINI-3-PRO-PREVIEW2026.02 | 62.2 | |
| GPT-52026.02 | 61.8 | |
| GEMINI-2.5-PRO2026.02 | 57 | |
| Gemini-2.5-ProModel Type=Closed-source MLLM2026.03 | 57 | |
| GeoThinker QWEN3VL-8BActive Perception=true2026.02 | 55.9 | |
| GeoThinker QWEN2.5VL-7BActive Perception=true2026.02 | 54.8 | |
| SEED-1.62026.02 | 54.6 | |
| ACE-Brain-0-8BModel Type=Embodied Brain MLLM2026.03 | 53.1 | |
| VG-LLM-8BActive Perception=false2026.02 | 52.6 | |
| RoboBrain2.5-8BModel Type=Embodied Brain MLLM, Evaluation Protocol=Our evaluation framework2026.03 | 52.6 | |
| Pelican-VL-7BModel Type=Embodied Brain MLLM, Evaluation Protocol=Our evaluation framework2026.03 | 52.3 | |
| VeBrain-7BModel Type=Embodied Brain MLLM, Evaluation Protocol=Our evaluation framework2026.03 | 51.4 | |
| VG-LLM-8B*Active Perception=false, Training Setting=S1+S22026.02 | 50.5 | |
| InternVL3.5-8BModel Type=Open-source general-purpose MLLM, Evaluation Protocol=Our evaluation framework2026.03 | 50.1 | |
| VG-LLM-4BActive Perception=false2026.02 | 49.8 | |
| RoboBrain2.0-7BModel Type=Embodied Brain MLLM, Evaluation Protocol=Our evaluation framework2026.03 | 49.2 | |
| Vlaser-8BModel Type=Embodied Brain MLLM, Evaluation Protocol=Our evaluation framework2026.03 | 47.5 | |
| QWEN3-VL-8B-INSTRUCT2026.02 | 45.8 | |
| Qwen3-VL-8B-Inst.Model Type=Open-source general-purpose MLLM2026.03 | 45.8 | |
| MiMo-Embodied-7BModel Type=Embodied Brain MLLM2026.03 | 44.8 | |
| INTERNVL3-8B2026.02 | 41.1 | |
| InternVL3-8BModel Type=Open-source general-purpose MLLM2026.03 | 41.1 | |
| VST-7B-SFT2026.02 | 39.6 | |
| GPT-4oModel Type=Closed-source MLLM2026.03 | 37.8 | |
| QWEN2.5-VL-7B-INSTRUCT2026.02 | 37.6 | |
| MiMo-VL-7BModel Type=Open-source general-purpose MLLM2026.03 | 37.6 | |
| BAGEL-7B-MOT2026.02 | 37 | |
| VST-3B-SFT2026.02 | 35.8 | |
| QWEN3-VL-2B-INSTRUCT2026.02 | 35.6 | |
| Qwen3-VL-2B-Inst.Model Type=Open-source general-purpose MLLM2026.03 | 35.6 | |
| QWEN2.5-VL-3B-INSTRUCT2026.02 | 33.1 | |
| CAMBRIAN-S-7B2026.02 | 33 | |
| Qwen2.5-VL-7B-Inst.Model Type=Open-source general-purpose MLLM2026.03 | 31.4 | |
| VLM-3R-7BActive Perception=false2026.02 | 31.3 | |
| INTERNVL3-2B2026.02 | 30 | |
| CAMBRIAN-S-3B2026.02 | 28.3 | |
| SPATIALLADDER-3B2026.02 | 27.9 | |
| Random Choice2026.02 | 0 |