Spatial relationship understanding on EmbSpatial
62.09AccuracyGemma3-27B-IT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemma3-27B-ITModel scale=27B2026.06 | 62.09 | 72.12 | |
| Thinking with Grounding PointModel scale=4B, Grounding interface=Point, Grounding reward=w/2026.06 | 60.88 | 83.1 | |
| Thinking with Grounding PointModel scale=4B, Grounding interface=Point, Grounding reward=w/o2026.06 | 60.25 | 83.21 | |
| Thinking with Grounding BoxModel scale=4B, Grounding interface=Box, Grounding reward=w/2026.06 | 59.93 | 82.66 | |
| Thinking with Grounding BoxModel scale=4B, Grounding interface=Box, Grounding reward=w/o2026.06 | 57.62 | 81.46 | |
| Gemma3-12B-ITModel scale=12B2026.06 | 56.68 | 65.14 | |
| Gemma3-4B-ITModel scale=4B2026.06 | 49.13 | 63.79 | |
| Non-grounded ThinkingModel scale=4B, Grounding interface=None2026.06 | 20.54 | 42.53 |