Spatial Reasoning on Metro-Spatial-QA
51.9Measurement AccuracyMosaicThinker
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MosaicThinkerModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 51.9 | 50.7 | 51.2 | |
| MosaicThinkerDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 49.2 | 54 | 52.6 | |
| Direct InputDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 45.6 | 47.3 | 45.9 | |
| MosaicThinkerHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 45.5 | 40.1 | 42.8 | |
| APCDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 45.4 | 50.6 | 48.2 | |
| APCModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 44.8 | 46.1 | 45.4 | |
| Scene Graph ReconstructionDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 44.7 | 49.2 | 47.3 | |
| Video-CoTDevice=Jetson Orion, Model=Qwen3VL-32B-4bit2026.02 | 44.2 | 48 | 46.1 | |
| Scene Graph ReconstructionModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 42.2 | 42.3 | 42.2 | |
| Video-CoTModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 41.5 | 41.6 | 41.5 | |
| APCHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 40.7 | 38.9 | 39.3 | |
| Direct InputModel=InternVL3-8B, Device=NVidia Jetson Orion2026.02 | 40.1 | 41.2 | 40.7 | |
| Scene Graph ReconstructionHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 38.4 | 38.1 | 38.2 | |
| Video-CoTHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 37.8 | 38.3 | 38.1 | |
| Direct InputHardware=Oneplus 12R Smartphone, VLM Backbone=Qwen-2.5-VL-3B2026.02 | 37.3 | 37.5 | 37.4 |