Spatial Reasoning on OmniSpatial (test)
95.24Dyn. ScoreHuman
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HumanCategory=Human Evaluation2025.06 | 95.24 | 94.36 | 92.63 | 93.48 | 87.86 | |
| GCAParadigm=Tool-Integrated Spatial Agents2025.11 | 73.6 | 58.6 | 65.1 | — | — | |
| Gemini-2.5-ProParadigm=Baseline Foundation VLMs2025.11 | 70.7 | 44.6 | 55.8 | — | — | |
| Gemini-2.5-pro-preview-05-06Rank=2, Category=Reasoning Models2025.06 | 69.48 | 49.96 | 55.19 | 67.38 | 39.07 | |
| o3-2025-04-16Rank=1, Category=Reasoning Models2025.06 | 69.03 | 57.88 | 56.33 | 65.07 | 34.95 | |
| Gemini-2.5-flash-05-20-thinkingRank=3, Category=Reasoning Models2025.06 | 67.5 | 49.2 | 53.16 | 63.91 | 35.59 | |
| o4-mini-2025-04-16Rank=3, Category=Reasoning Models2025.06 | 66.4 | 51.73 | 52.77 | 65.05 | 35.4 | |
| o1-2024-12-17Rank=6, Category=Reasoning Models2025.06 | 66.3 | 48.58 | 50.36 | 60.49 | 33.14 | |
| GPT-4.1-2025-04-14Rank=2, Category=Proprietary Models2025.06 | 65.48 | 50.22 | 51.78 | 61.84 | 30.91 | |
| Gemini-2.5-flash-preview-05-20Rank=1, Category=Proprietary Models2025.06 | 65.14 | 48.37 | 52.12 | 67.49 | 36.16 | |
| InternVL3-38BRank=2, Category=Open-source Models2025.06 | 63.5 | 47.47 | 48.48 | 54.48 | 29.21 | |
| InternVL3-78BRank=1, Category=Open-source Models2025.06 | 63.45 | 49.62 | 49.33 | 55.64 | 28.91 | |
| GPT-4o-2024-11-20Rank=5, Category=Proprietary Models2025.06 | 61.39 | 51.74 | 47.81 | 54.31 | 25.88 | |
| Qwen3-VL-ThinkingParadigm=Baseline Foundation VLMs2025.11 | 60.5 | 43.9 | 51 | — | — | |
| GPT-4.1-mini-2025-04-14Rank=4, Category=Proprietary Models2025.06 | 60.42 | 50.47 | 48.87 | 58.54 | 29.73 | |
| SpatialLLMParadigm=Training-based Spatial VLMs2025.11 | 59.6 | 42.9 | 49.5 | — | — | |
| Qwen-VL2.5-72BRank=3, Category=Open-source Models2025.06 | 59.25 | 48.19 | 47.85 | 54.52 | 29.61 | |
| Qwen-VL2.5-32BRank=4, Category=Open-source Models2025.06 | 59.08 | 48.59 | 47.36 | 58.32 | 26.94 | |
| Gemini-2.0-flash-expRank=2, Category=Proprietary Models2025.06 | 58.95 | 50.41 | 48.4 | 58.09 | 27.32 | |
| GPT-4oParadigm=Baseline Foundation VLMs2025.11 | 58.7 | 46.2 | 51.5 | — | — | |
| GLM-4.5VParadigm=Baseline Foundation VLMs2025.11 | 58.6 | 47.2 | 52.1 | — | — | |
| Claude-3-7-sonnet-20250219-thinkingRank=7, Category=Reasoning Models2025.06 | 58.47 | 47.84 | 48.62 | 59.65 | 29.2 | |
| InternVL3-14BRank=5, Category=Open-source Models2025.06 | 57.25 | 45.96 | 45.94 | 51.2 | 28.15 | |
| Claude-3-7-sonnet-20250219Rank=5, Category=Proprietary Models2025.06 | 56.76 | 48.28 | 47.53 | 59.87 | 28.94 | |
| Gemma-3-27BRank=7, Category=Open-source Models2025.06 | 56.27 | 43.58 | 44.75 | 53.62 | 28.44 | |
| LLaVA-onevision-qwen2-72BRank=6, Category=Open-source Models2025.06 | 56.22 | 49.14 | 45.66 | 57.14 | 24.24 | |
| Gemma-3-12BRank=8, Category=Open-source Models2025.06 | 54.48 | 44.72 | 43.71 | 49.06 | 23.41 | |
| Claude-3-5-sonnet-20241022Rank=6, Category=Proprietary Models2025.06 | 54.31 | 48.1 | 46.86 | 59.86 | 29.17 | |
| SoFar-Qwen2.5VL-3BRank=1, Category=Specialized Spatial Reasoning Models2025.06 | 53.83 | 49.95 | 45.14 | 53.33 | 27.31 | |
| SpaceRParadigm=Training-based Spatial VLMs2025.11 | 53.5 | 40.5 | 46 | — | — | |
| GPT-4o-mini-2024-07-18Rank=8, Category=Proprietary Models2025.06 | 53.12 | 44.18 | 42.64 | 47.64 | 25.95 | |
| Gemini-2.0-flash-lite-02-05Rank=8, Category=Proprietary Models2025.06 | 52.95 | 47.36 | 44.03 | 54.34 | 26.44 | |
| TIGeRParadigm=Tool-Integrated Spatial Agents2025.11 | 52.9 | 45.7 | 49.8 | — | — | |
| GPT-4.1-nano-2025-04-14Rank=9, Category=Proprietary Models2025.06 | 52.38 | 41.52 | 42.62 | 46.09 | 27.25 | |
| Qwen-VL2.5-3BRank=10, Category=Open-source Models2025.06 | 51.46 | 41.18 | 40.3 | 44.38 | 28.02 | |
| SpaceThinker-Qwen2.5VL-3BRank=2, Category=Specialized Spatial Reasoning Models2025.06 | 50.45 | 41.41 | 40.42 | 39.15 | 26.16 | |
| Video-R1Paradigm=Training-based Spatial VLMs2025.11 | 50 | 44.2 | 46.7 | — | — | |
| RoboBrain-2.0Paradigm=Training-based Spatial VLMs2025.11 | 49.4 | 42.2 | 45.2 | — | — | |
| SpaceQwen2.5-VL-3BRank=3, Category=Specialized Spatial Reasoning Models2025.06 | 49 | 47.44 | 40.25 | 41.01 | 27.85 | |
| Qwen-VL2.5-7BRank=12, Category=Open-source Models2025.06 | 46.73 | 45.02 | 39.18 | 46.48 | 30.27 | |
| InternVL3-8BRank=9, Category=Open-source Models2025.06 | 46.65 | 47.93 | 41.6 | 48.25 | 26.79 | |
| SpatialLadderParadigm=Training-based Spatial VLMs2025.11 | 46.5 | 43.1 | 44.5 | — | — | |
| Gemma-3-4BRank=11, Category=Open-source Models2025.06 | 45.8 | 44.84 | 39.79 | 40.15 | 24.12 | |
| InternVL3-2BRank=13, Category=Open-source Models2025.06 | 45.29 | 41.2 | 37.98 | 41.28 | 25.19 | |
| LLavA-1.5-vicuna-7BRank=15, Category=Open-source Models2025.06 | 42.84 | 42.13 | 34.97 | 35.14 | 26.59 | |
| RoboPoint-vicuna-v1.5-7B-loraRank=6, Category=Specialized Spatial Reasoning Models2025.06 | 42.82 | 43.29 | 35.85 | 37.57 | 26.3 | |
| RoboPoint-vicuna-v1.5-13BRank=5, Category=Specialized Spatial Reasoning Models2025.06 | 41.91 | 40.06 | 34.6 | 35.85 | 25.93 | |
| SpaceMantis-13BRank=6, Category=Specialized Spatial Reasoning Models2025.06 | 41.81 | 42.25 | 36.36 | 36.3 | 23.33 | |
| LLaVA-onevision-qwen2-7BRank=14, Category=Open-source Models2025.06 | 40.7 | 40.19 | 35.68 | 34.76 | 25.73 | |
| SpatialBot-3BRank=6, Category=Specialized Spatial Reasoning Models2025.06 | 40.7 | 40.19 | 35.68 | 34.76 | 25.73 | |
| VLaserParadigm=Training-based Spatial VLMs2025.11 | 39.1 | 42.6 | 41.1 | — | — | |
| VILASRParadigm=Training-based Spatial VLMs2025.11 | 37.5 | 42.2 | 40.2 | — | — | |
| Spatial-MLLMParadigm=Training-based Spatial VLMs2025.11 | 37.2 | 42.1 | 40 | — | — |