Spatial Reasoning on STI-Bench
37.5D-Measure ScoreGround Truth Semantic Map
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Ground Truth Semantic MapDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 37.5 | 34.4 | 35.9 | |
| Ground Truth Semantic MapModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 36.8 | 39 | 38.3 | |
| MosaicThinkerDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 36.4 | 33.9 | 34.8 | |
| MosaicThinkerModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 35.3 | 38.9 | 38.1 | |
| APCDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 32 | 30.8 | 31.4 | |
| Ground Truth Semantic MapHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 31.3 | 32.9 | 32.1 | |
| APCModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 31.1 | 35.7 | 34.4 | |
| MosaicThinkerHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 30.8 | 31.8 | 31.6 | |
| Scene Graph ReconstructionDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 30.7 | 31.9 | 31.3 | |
| Video-CoTDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 30 | 31.8 | 30.9 | |
| Direct InputDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 29.5 | 32.2 | 30.8 | |
| Scene Graph ReconstructionModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 29.2 | 35.1 | 32.2 | |
| APCHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 29.1 | 30.2 | 29.6 | |
| Video-CoTModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 29.1 | 35 | 32.5 | |
| Direct InputModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 28.5 | 34.9 | 31.8 | |
| Scene Graph ReconstructionHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 27.6 | 28.4 | 28 | |
| Video-CoTHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 26.5 | 27.9 | 27.8 | |
| Direct InputHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 25.8 | 27.4 | 26.2 |