Spatial Mental Modeling on SAT (synthesized)
95.4EgoMSpatialDreamer
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SpatialDreamerCategory=Spatial MLLM, Backbone=Qwen2.5-VL-7B-Instruct2025.12 | 95.4 | 93.1 | 91.8 | 94.9 | 89.8 | 92.5 | |
| MindJourney (o1, SVC)Category=Test-time Scaling Method, Base Model=o1, World Model=SVC2025.12 | 87.1 | 80.4 | 72.3 | 89.3 | 70.1 | 78.6 | |
| MindJourney (o1, SWM)Category=Test-time Scaling Method, Base Model=o1, World Model=SWM2025.12 | 82.4 | 80.4 | 76.6 | 88.1 | 60.7 | 76.8 | |
| SAT (LLaVA-Video-7B)Category=Spatial MLLM, Backbone=LLaVA-Video-7B2025.12 | 79.6 | 80.4 | 85.3 | 56.4 | 88.4 | 78 | |
| o1Category=Generalist MLLM2025.12 | 78 | 85.9 | 65.4 | 86 | 54.6 | 72.4 | |
| InternVL3-14BCategory=Generalist MLLM, Size=14B2025.12 | 77.6 | 85.9 | 53.3 | 84.5 | 20.6 | 61.6 | |
| GPT-4.1Category=Generalist MLLM2025.12 | 75.3 | 89 | 57.8 | 78.3 | 41.5 | 66.4 | |
| Gemini 1.5-flashCategory=Generalist MLLM2025.12 | 67.1 | 33.1 | 52.9 | 64 | 32.7 | 50 | |
| GPT-4oCategory=Generalist MLLM2025.12 | 64.7 | 86.8 | 51.9 | 68.7 | 43.4 | 61 | |
| SAT (LLaVA-1.5-13B)Category=Spatial MLLM, Backbone=LLaVA-1.5-13B2025.12 | 61.7 | 90.2 | 91.4 | 96.8 | 98.5 | 87.7 | |
| Gemini 1.5-proCategory=Generalist MLLM2025.12 | 57.7 | 29.8 | 55.5 | 56.9 | 49.5 | 49.9 | |
| LLaVA-Video-7BCategory=Generalist MLLM, Size=7B2025.12 | 56.4 | 82.7 | 48 | 52.9 | 47.1 | 57.4 | |
| GPT-4VCategory=Generalist MLLM2025.12 | 54.7 | 32.7 | 52 | 50.5 | 34.2 | 44.8 | |
| Robopoint-13BCategory=Spatial MLLM, Size=13B2025.12 | 50.2 | 69.4 | 48.8 | 72.6 | 25.5 | 53.3 | |
| LLaVA-1.5-13BCategory=Generalist MLLM, Size=13B2025.12 | 46.6 | 73.8 | 49.7 | 45.6 | 39.9 | 51.1 |