Situated Question Answering on SQA3D (test)
62.1EM (What)OneCanvas
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| OneCanvas2026.06 | 62.1 | — | — | 76.2 | 61.1 | 75.4 | 74.4 | 70.5 | 65.3 | 68.4 | |
| Ross3D2026.06 | 56 | — | — | 79.8 | 60.6 | 70.4 | 55.3 | 60.1 | 63 | 65.7 | |
| GPT4Scene-HDM2026.06 | 55.9 | — | — | 69.9 | 50.8 | 68.7 | 53.3 | 60.4 | 59.4 | 62.4 | |
| SR-3D2026.06 | 55 | — | — | 76.4 | 59.8 | 71.6 | 54.7 | 61.1 | 62.2 | — | |
| SpaceMind2026.06 | 54.1 | — | — | 74.8 | 61.7 | 71 | 51.9 | 53.6 | 61.1 | 63.8 | |
| SpaR3D-MoEModel Category=Video-Input Models2026.07 | 51.6 | 58.3 | — | 72.6 | 56.3 | 69.2 | 52.1 | 54.2 | — | — | |
| Video-3D LLM2026.06 | 51.1 | — | — | 72.4 | 55.5 | 69.8 | 51.3 | 56 | 58.6 | 60.8 | |
| Video-3D LLMModel Category=3D/2.5D-Input Models2026.07 | 51.1 | 58.6 | — | 72.4 | 55.5 | 69.8 | 51.3 | 56 | — | — | |
| Spatial-MLLM-4BModel Category=Video-Input Models2026.07 | 45.9 | 55.9 | — | 71.6 | 55.1 | 69.5 | 52 | 53 | — | — | |
| ChatScene2026.06 | 45.4 | — | — | 67 | 52 | 69.5 | 49.9 | 55 | 54.6 | 57.5 | |
| Chat-SceneModel Category=3D/2.5D-Input Models2026.07 | 45.4 | 54.6 | — | 67 | 52 | 69.5 | 49.9 | 55 | — | — | |
| LLaVA-Video-7BModel Category=Video-Input Models2026.07 | 42.7 | 48.5 | — | 56.3 | 47.5 | 55.3 | 50.1 | 47.2 | — | — | |
| Qwen2.5-VL-72BModel Category=Video-Input Models2026.07 | 41.7 | 47 | — | 56.3 | 41.5 | 55.6 | 44.5 | 48 | — | — | |
| Scene-LLM2026.06 | 40.9 | — | — | 69.1 | 45 | 70.8 | 47.2 | 52.3 | 54.2 | 56.2 | |
| Scene-LLMModel Category=3D/2.5D-Input Models2026.07 | 40.9 | 54.2 | — | 69.1 | 45 | 70.8 | 47.2 | 52.3 | — | — | |
| Qwen2.5-VL-7BModel Category=Video-Input Models2026.07 | 39.7 | 46.5 | — | 56.6 | 41.1 | 55.9 | 47.6 | 47.2 | — | — | |
| LEO2026.06 | 39 | — | — | 63.9 | 44.9 | 66.2 | 47.7 | 51.1 | 50 | 52.4 | |
| PQ3D2026.06 | 37.1 | — | — | 61.3 | 44.5 | 60.9 | 47 | 45.1 | 47.1 | 49.3 | |
| SIG3D2026.06 | 35.6 | — | — | 67.2 | 48.5 | 71.4 | 49.1 | 45.8 | 52.6 | 54.4 | |
| 3D-VisTA2026.06 | 34.8 | — | — | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | 48.5 | 50.9 | |
| 3D-VistaModel Category=Task-Specific Models2026.07 | 34.8 | 48.5 | — | 63.3 | 45.4 | 69.8 | 47.2 | 48.1 | — | — | |
| Qwen2.5-VL-3BModel Category=Video-Input Models2026.07 | 34.8 | 43.4 | — | 52.1 | 39.8 | 52.7 | 45.6 | 47 | — | — | |
| SQA3DModel Category=Task-Specific Models2026.07 | 31.6 | 46.6 | — | 63.8 | 46 | 69.5 | 43.9 | 45.3 | — | — | |
| 3D-LLaVAModality=PC, Model Type=3D LMM2025.01 | — | 54.5 | 56.6 | — | — | — | — | — | — | — | |
| 3D-VisTAModality=PC, Model Type=Specialist Model2025.01 | — | 48.5 | — | — | — | — | — | — | — | — | |
| Chat-SceneModality=PC+I, Model Type=3D LMM2025.01 | — | 54.6 | 57.5 | — | — | — | — | — | — | — | |
| LEOModality=PC+I, Model Type=3D LMM2025.01 | — | 50 | 52.4 | — | — | — | — | — | — | — | |
| Loc3R-VLM2026.06 | — | — | — | — | — | — | — | — | 62.8 | 65 | |
| ScanQAModality=PC, Model Type=Specialist Model2025.01 | — | 46.6 | — | — | — | — | — | — | — | — | |
| Scene-LLMModality=PC+I, Model Type=Finetuned 3D LMM2025.01 | — | 54.2 | — | — | — | — | — | — | — | — | |
| Scene-LLMModality=PC+I, Model Type=3D LMM2025.01 | — | 53.6 | — | — | — | — | — | — | — | — | |
| VLM-3R2026.06 | — | — | — | — | — | — | — | — | 60.7 | 63.4 |