Spatial Mental Modeling on SAT-Real
93.9AVGSpatialDreamer
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SpatialDreamerCategory=Spatial MLLM, Backbone=Qwen2.5-VL-7B-Instruct2025.12 | 93.9 | 100 | 90.5 | 93.5 | 92.7 | 90.3 | |
| MindJourney (o1, SWM)Category=Test-time Scaling Method, Base Model=o1, World Model=SWM2025.12 | 84.7 | 95.7 | 82.6 | 83.8 | 88.2 | 75.8 | |
| Robix-32BCategory=Spatial MLLM, Size=32B2025.12 | 79.6 | — | — | — | — | — | |
| MindJourney (o1, SVC)Category=Test-time Scaling Method, Base Model=o1, World Model=SVC2025.12 | 77.3 | 100 | 65.2 | 78.4 | 82.4 | 63.7 | |
| o1Category=Generalist MLLM2025.12 | 74.6 | 78.3 | 82.6 | 73 | 73.5 | 69.7 | |
| GPT-4.1Category=Generalist MLLM2025.12 | 74 | 95.7 | 73.9 | 78.3 | 88.2 | 39.4 | |
| ViSABeam depth (gamma)=1, top-k (k)=42025.12 | 72.67 | 95.65 | 65.22 | 75.68 | 79.41 | 51.52 | |
| MindJourneyBeam depth (gamma)=2, top-k (k)=22025.12 | 70.67 | 73.91 | 73.91 | 75.68 | 76.47 | 54.55 | |
| ViSABeam depth (gamma)=1, top-k (k)=32025.12 | 69.33 | 82.61 | 65.21 | 78.38 | 76.47 | 45.45 | |
| ViSABeam depth (gamma)=2, top-k (k)=12025.12 | 69.33 | 73.91 | 65.22 | 81.08 | 70.59 | 54.55 | |
| ViSABeam depth (gamma)=1, top-k (k)=22025.12 | 68.67 | 69.57 | 69.57 | 75.68 | 85.29 | 42.42 | |
| ViSABeam depth (gamma)=2, top-k (k)=32025.12 | 68.67 | 69.57 | 73.91 | 81.08 | 70.59 | 48.48 | |
| RandomBeam depth (gamma)=2, top-k (k)=42025.12 | 67.33 | 73.91 | 56.52 | 78.38 | 73.53 | 51.52 | |
| MindJourneyBeam depth (gamma)=1, top-k (k)=22025.12 | 67.33 | 60.87 | 65.22 | 75.68 | 79.41 | 51.52 | |
| ViSABeam depth (gamma)=2, top-k (k)=22025.12 | 67.33 | 78.26 | 65.21 | 62.16 | 79.41 | 54.55 | |
| MindJourneyBeam depth (gamma)=2, top-k (k)=12025.12 | 66.67 | 69.57 | 69.57 | 70.27 | 79.41 | 45.45 | |
| MindJourneyBeam depth (gamma)=2, top-k (k)=42025.12 | 66.67 | 69.57 | 65.21 | 72.97 | 79.41 | 45.45 | |
| ViSABeam depth (gamma)=2, top-k (k)=42025.12 | 66.67 | 65.21 | 56.52 | 72.97 | 82.35 | 51.52 | |
| RandomBeam depth (gamma)=1, top-k (k)=22025.12 | 66 | 73.91 | 56.52 | 70.27 | 76.47 | 51.52 | |
| ViSABeam depth (gamma)=1, top-k (k)=12025.12 | 65.33 | 65.22 | 65.22 | 72.97 | 79.41 | 42.42 | |
| Gemini 1.5-proCategory=Generalist MLLM2025.12 | 64.8 | — | — | — | — | — | |
| MindJourneyBeam depth (gamma)=1, top-k (k)=32025.12 | 64.67 | 73.91 | 65.22 | 64.86 | 73.53 | 48.48 | |
| MindJourneyBeam depth (gamma)=1, top-k (k)=42025.12 | 64.67 | 73.91 | 65.22 | 64.87 | 76.47 | 45.45 | |
| RandomBeam depth (gamma)=1, top-k (k)=32025.12 | 64 | 82.61 | 56.52 | 62.16 | 76.47 | 45.45 | |
| SAT (LLaVA-Video-7B)Category=Spatial MLLM, Backbone=LLaVA-Video-7B2025.12 | 63.4 | — | — | — | — | — | |
| RandomBeam depth (gamma)=1, top-k (k)=12025.12 | 63.33 | 65.22 | 56.52 | 78.38 | 70.59 | 42.42 | |
| RandomBeam depth (gamma)=1, top-k (k)=42025.12 | 63.33 | 69.57 | 60.87 | 54.05 | 76.47 | 57.58 | |
| MindJourneyBeam depth (gamma)=1, top-k (k)=12025.12 | 63.33 | 65.22 | 52.17 | 72.97 | 76.47 | 45.45 | |
| MindJourneyBeam depth (gamma)=2, top-k (k)=32025.12 | 63.33 | 52.17 | 65.21 | 64.86 | 76.47 | 54.55 | |
| RandomBeam depth (gamma)=2, top-k (k)=12025.12 | 61.33 | 52.17 | 60.87 | 70.27 | 79.41 | 39.4 | |
| GPT-4oCategory=Generalist MLLM2025.12 | 60.3 | 56.5 | 85 | 50 | 64 | 45 | |
| RandomBeam depth (gamma)=2, top-k (k)=32025.12 | 59.33 | 65.22 | 52.17 | 67.57 | 70.59 | 39.4 | |
| InternVL3-14BCategory=Generalist MLLM, Size=14B2025.12 | 59.3 | 56.5 | 69.5 | 54 | 73.5 | 45.4 | |
| RandomBeam depth (gamma)=2, top-k (k)=22025.12 | 58.67 | 52.17 | 52.17 | 59.46 | 76.47 | 48.48 | |
| Gemini 1.5-flashCategory=Generalist MLLM2025.12 | 57.6 | — | — | — | — | — | |
| SAT (LLaVA-1.5-13B)Category=Spatial MLLM, Backbone=LLaVA-1.5-13B2025.12 | 54.9 | — | — | — | — | — | |
| LLaVA-Video-7BCategory=Generalist MLLM, Size=7B2025.12 | 53.5 | — | — | — | — | — | |
| GPT-4VCategory=Generalist MLLM2025.12 | 50.7 | — | — | — | — | — | |
| Robopoint-13BCategory=Spatial MLLM, Size=13B2025.12 | 46.6 | — | — | — | — | — | |
| LLaVA-1.5-13BCategory=Generalist MLLM, Size=13B2025.12 | 41.6 | — | — | — | — | — | |
| InternVL3-14Bbaseline=true2025.12 | 41.33 | 52.17 | 69.57 | 32.43 | 32.35 | 33.33 |