# Spatial Reasoning on VSI-Bench Vanilla regime Source: [Spatial Reasoning on VSI-Bench Vanilla regime]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:36:23.356Z Evaluation of model performance on the VSI-Bench dataset under the Vanilla regime, focusing on Spatial Reasoning capabilities and reporting metrics including AVG, ABS. DIST., and ROUTE PLAN. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Spatial Reasoning | | Dataset (including any stated split) | VSI-Bench Vanilla regime | | Metric | Avg Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 20 of 20 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Avg Score](): Higher is better; unit: PERCENT. - [Object Count](): Higher is better; unit: SCALAR. - [Absolute Distance](): Lower is better; unit: SCALAR. - [Object Size](): Lower is better; unit: SCALAR. - [Room Size](): Higher is better; unit: SCALAR. - [Relative Distance](): Lower is better; unit: PERCENT. - [Relative Direction Error](): Lower is better; unit: SCALAR. - [Route Plan Success](): Higher is better; unit: PERCENT. - [Approach Order Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Avg Score | Result date | Report paper | | --- | --- | --- | --- | | GeoThinker Qwen2.5VL-7B | 50.5 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | VG-LLM-8B | 49.7 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | GeoThinker Qwen2.5VL-3B | 48.9 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | Spatial-MLLM-4B | 48.4 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | VG-LLM-4B | 46.7 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | Gemini-1.5 Pro | 45.4 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | SpatialLadder-3B | 44.8 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | SPAR-8B | 44.1 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | Gemini-1.5 Flash | 42.1 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | LLaVA-NeXT-Video-72B | 40.9 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | LLaVA-OneVision-72B | 40.2 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | InternVL2-40B | 36 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | LLaVA-NeXT-Video-7B | 35.6 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | InternVL2-8B | 34.6 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | Chance Level (Frequency) | 34 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | GPT-4O | 34 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | LLaVA-OneVision-7B | 32.4 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | Qwen2.5VL-7B | 29.3 | Not recorded | 698575e4c97a0b3e3b5d5f2a | | Qwen2.5VL-3B | 28.6 | Not recorded | 698575e4c97a0b3e3b5d5f2a | ### GeoThinker Qwen2.5VL-7B Avg Score: 50.5. Result ID: 6997383f5698665ec4883e60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5VL-7B","Active Perception":true}. This record does not assert independent reproduction. ### VG-LLM-8B Avg Score: 49.7. Result ID: 6997383f5698665ec4883e5e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-Source Spatial Intelligence Model","Parameters":"8B","Active Perception":false}. This record does not assert independent reproduction. ### GeoThinker Qwen2.5VL-3B Avg Score: 48.9. Result ID: 6997383f5698665ec4883e5f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5VL-3B","Active Perception":true}. This record does not assert independent reproduction. ### Spatial-MLLM-4B Avg Score: 48.4. Result ID: 6997383f5698665ec4883e5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-Source Spatial Intelligence Model","Parameters":"4B","Active Perception":false}. This record does not assert independent reproduction. ### VG-LLM-4B Avg Score: 46.7. Result ID: 6997383f5698665ec4883e5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-Source Spatial Intelligence Model","Parameters":"4B","Active Perception":false}. This record does not assert independent reproduction. ### Gemini-1.5 Pro Avg Score: 45.4. Result ID: 6997383f5698665ec4883e51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Proprietary Model (API)"}. This record does not assert independent reproduction. ### SpatialLadder-3B Avg Score: 44.8. Result ID: 6997383f5698665ec4883e5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-Source Spatial Intelligence Model","Parameters":"3B"}. This record does not assert independent reproduction. ### SPAR-8B Avg Score: 44.1. Result ID: 6997383f5698665ec4883e5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-Source Spatial Intelligence Model","Parameters":"8B"}. This record does not assert independent reproduction. ### Gemini-1.5 Flash Avg Score: 42.1. Result ID: 6997383f5698665ec4883e50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Proprietary Model (API)"}. This record does not assert independent reproduction. ### LLaVA-NeXT-Video-72B Avg Score: 40.9. Result ID: 6997383f5698665ec4883e55. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"72B"}. This record does not assert independent reproduction. ### LLaVA-OneVision-72B Avg Score: 40.2. Result ID: 6997383f5698665ec4883e53. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"72B"}. This record does not assert independent reproduction. ### InternVL2-40B Avg Score: 36. Result ID: 6997383f5698665ec4883e57. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"40B"}. This record does not assert independent reproduction. ### LLaVA-NeXT-Video-7B Avg Score: 35.6. Result ID: 6997383f5698665ec4883e54. [Reporting paper]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"7B"}. This record does not assert independent reproduction. ### InternVL2-8B Avg Score: 34.6. Result ID: 6997383f5698665ec4883e56. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"8B"}. This record does not assert independent reproduction. ### Chance Level (Frequency) Avg Score: 34. Result ID: 6997383f5698665ec4883e4e. [Reporting paper]() Recorded evaluation configuration: {"Type":"Baseline"}. This record does not assert independent reproduction. ### GPT-4O Avg Score: 34. Result ID: 6997383f5698665ec4883e4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Proprietary Model (API)"}. This record does not assert independent reproduction. ### LLaVA-OneVision-7B Avg Score: 32.4. Result ID: 6997383f5698665ec4883e52. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"7B"}. This record does not assert independent reproduction. ### Qwen2.5VL-7B Avg Score: 29.3. Result ID: 6997383f5698665ec4883e59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"7B"}. This record does not assert independent reproduction. ### Qwen2.5VL-3B Avg Score: 28.6. Result ID: 6997383f5698665ec4883e58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Category":"Open-Sourced Model","Parameters":"3B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.