Compositional Reasoning on Lego Puzzles
57.7AccuracyGPT-4o-0513
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4o-05132025.11 | 57.7 | |
| Claude-3.5-Sonnet-06202025.11 | 53.6 | |
| SpatialThinker-7B (Ours)Training Data=SpatialThinkerVQA2025.11 | 37.7 | |
| Qwen2.5-VL-7B + SFTTraining Data=SpatialThinkerVQA, Training Protocol=SFT2025.11 | 36.6 | |
| Qwen2.5-VL-7B2025.11 | 35.8 | |
| Qwen2.5-VL-3B + SFTTraining Data=SpatialThinkerVQA, Training Protocol=SFT2025.11 | 34.7 | |
| SpatialThinker-3B (Ours)Training Data=SpatialThinkerVQA2025.11 | 33.9 | |
| VLAA-Thinker-7B2025.11 | 33.4 | |
| SpaceOm2025.11 | 32 | |
| SpaceThinker2025.11 | 31.5 | |
| Qwen2.5-VL-3B2025.11 | 29.9 | |
| Qwen2.5-VL-7B + Vanilla GRPOTraining Data=SpatialThinkerVQA, Training Protocol=Vanilla GRPO2025.11 | 29.7 | |
| Qwen2.5-VL-3B + Vanilla GRPOTraining Data=SpatialThinkerVQA, Training Protocol=Vanilla GRPO2025.11 | 27 |