Spatial Reasoning on VSI-Bench Vanilla regime
50.5Avg ScoreGeoThinker Qwen2.5VL-7B
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GeoThinker Qwen2.5VL-7BBackbone=Qwen2.5VL-7B, Active Perception=true2026.02 | 50.5 | 69.5 | 38.5 | 57.9 | 62.2 | 45.2 | 46.2 | 31.4 | 52.6 | |
| VG-LLM-8BModel Category=Open-Source Spatial Intelligence Model, Parameters=8B, Active Perception=false2026.02 | 49.7 | 68.1 | 38.7 | 59 | 61.1 | 45.5 | 44.9 | 26.8 | 53.4 | |
| GeoThinker Qwen2.5VL-3BBackbone=Qwen2.5VL-3B, Active Perception=true2026.02 | 48.9 | 68.5 | 36.1 | 57.3 | 62.5 | 43.7 | 47.9 | 34.5 | 40.9 | |
| Spatial-MLLM-4BModel Category=Open-Source Spatial Intelligence Model, Parameters=4B, Active Perception=false2026.02 | 48.4 | 65.3 | 34.8 | 63.1 | 45.1 | 41.3 | 46.2 | 33.5 | 46.3 | |
| VG-LLM-4BModel Category=Open-Source Spatial Intelligence Model, Parameters=4B, Active Perception=false2026.02 | 46.7 | 67.6 | 37.6 | 55.2 | 52.5 | 48 | 44.7 | 31.9 | 35.5 | |
| Gemini-1.5 ProModel Category=Proprietary Model (API)2026.02 | 45.4 | 56.2 | 30.9 | 64.1 | 43.6 | 51.3 | 46.3 | 36 | 34.6 | |
| SpatialLadder-3BModel Category=Open-Source Spatial Intelligence Model, Parameters=3B2026.02 | 44.8 | — | — | — | — | — | — | — | — | |
| SPAR-8BModel Category=Open-Source Spatial Intelligence Model, Parameters=8B2026.02 | 44.1 | — | — | — | — | — | — | — | — | |
| Gemini-1.5 FlashModel Category=Proprietary Model (API)2026.02 | 42.1 | 49.8 | 30.8 | 53.5 | 54.4 | 37.7 | 41 | 31.5 | 37.8 | |
| LLaVA-NeXT-Video-72BModel Category=Open-Sourced Model, Parameters=72B2026.02 | 40.9 | 48.9 | 22.8 | 57.4 | 35.3 | 42.4 | 36.7 | 35 | 48.6 | |
| LLaVA-OneVision-72BModel Category=Open-Sourced Model, Parameters=72B2026.02 | 40.2 | 43.5 | 23.9 | 57.6 | 37.5 | 42.5 | 39.9 | 32.5 | 44.6 | |
| InternVL2-40BModel Category=Open-Sourced Model, Parameters=40B2026.02 | 36 | 34.9 | 26.9 | 46.5 | 31.8 | 42.1 | 32.2 | 34 | 39.6 | |
| LLaVA-NeXT-Video-7BModel Category=Open-Sourced Model, Parameters=7B2026.02 | 35.6 | 48.5 | 14 | 47.8 | 24.2 | 43.5 | 42.4 | 34 | 30.6 | |
| InternVL2-8BModel Category=Open-Sourced Model, Parameters=8B2026.02 | 34.6 | 23.1 | 28.7 | 48.2 | 39.8 | 36.7 | 30.7 | 29.9 | 39.6 | |
| Chance Level (Frequency)Type=Baseline2026.02 | 34 | 62.1 | 32 | 29.9 | 33.1 | 25.1 | 47.9 | 28.4 | 25.2 | |
| GPT-4OModel Category=Proprietary Model (API)2026.02 | 34 | 46.2 | 5.3 | 43.8 | 38.2 | 37 | 41.3 | 31.5 | 28.5 | |
| LLaVA-OneVision-7BModel Category=Open-Sourced Model, Parameters=7B2026.02 | 32.4 | 47.7 | 20.2 | 47.4 | 12.3 | 42.5 | 35.2 | 29.4 | 24.4 | |
| Qwen2.5VL-7BModel Category=Open-Sourced Model, Parameters=7B2026.02 | 29.3 | 25.2 | 10.5 | 36.4 | 29.6 | 38.4 | 38 | 29.8 | 26.8 | |
| Qwen2.5VL-3BModel Category=Open-Sourced Model, Parameters=3B2026.02 | 28.6 | 32.7 | 19.5 | 17.3 | 25.1 | 37.3 | 44.9 | 30.4 | 21.8 | |
| Chance Level (Random)Type=Baseline2026.02 | — | — | — | — | — | 25 | 36.1 | 28.3 | 25 |