Spatial Reasoning on MINDCUBE 1.0 (test)
47.62Overall ScoreDeepSeek-VL2-Small
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DeepSeek-VL2-SmallModel Category=Open-Weight Multi Image Model2025.06 | 47.62 | 37 | 50.38 | 26.91 | |
| GPT-4o-2024-08-06Model Category=Proprietary Model2025.06 | 47.59 | 93.33 | 34.17 | 41.63 | |
| LLaVA-Onevision-7BModel Category=Open-Weight Multi Image Model2025.06 | 47.43 | 36.45 | 48.42 | 44.09 | |
| Gemini-1.5-pro-002Model Category=Proprietary Model2025.06 | 47.05 | 85.5 | 25.95 | 38.4 | |
| Gemma-2-9B-itModel Category=Open-Weight Multi Image Model2025.06 | 46.67 | 38.39 | 48.38 | 34.63 | |
| mPLUG-Owl3-7B-241101Model Category=Open-Weight Multi Image Model2025.06 | 44.85 | 37.84 | 47.11 | 26.91 | |
| Claude-3.5-Sonnet-20240620Model Category=Proprietary Model2025.06 | 44.75 | 48.42 | 44.21 | 47.62 | |
| LLaVA-Video-Qwen-7BModel Category=Open-Weight Multi Image Model2025.06 | 41.96 | 35.71 | 43.55 | 30.12 | |
| Mantis-8B (SigLip)Model Category=Open-Weight Multi Image Model2025.06 | 41.05 | 37.65 | 40.23 | 50.99 | |
| InternVL2-8BModel Category=Open-Weight Multi Image Model2025.06 | 37.5 | 26 | 42.03 | 36 | |
| RoboBrainModel Category=Spatial Model2025.06 | 37.38 | 35.8 | 38.28 | 29.53 | |
| Space-QwenModel Category=Spatial Model2025.06 | 33.28 | 38.02 | 33.71 | 26.32 | |
| Qwen2-VL-2B-InstructModel Category=Open-Weight Multi Image Model2025.06 | 33.21 | 37.37 | 33.26 | 30.34 | |
| Random (frequency)Model Category=Baseline2025.06 | 33.02 | 38.3 | 32.66 | 35.79 | |
| Random (chance)Model Category=Baseline2025.06 | 32.35 | 36.36 | 32.29 | 30.66 | |
| Spatial-MLLMModel Category=Spatial Model2025.06 | 32.06 | 38.39 | 20.92 | 32.82 | |
| LongVA-7BModel Category=Open-Weight Multi Image Model2025.06 | 29.46 | 35.89 | 29.55 | 24.88 | |
| Qwen2-VL-7B-InstructModel Category=Open-Weight Multi Image Model2025.06 | 29.26 | 38.76 | 29.5 | 21.35 | |
| SpaceMantisModel Category=Spatial Model2025.06 | 22.81 | 37.65 | 21.26 | 29.32 |