Spatial Reasoning on Satellite Imagery on SQuID Tier 1
53.52AccuracyQVLM
Evaluation Results
| Method | Links | |
|---|---|---|
| QVLMCode Generator=GPT-5, Segmentation Model=ConvNeXt, Protocol=Zero-shot, pass@1, Reasoning Config=Minimal effort without token limits2026.01 | 53.52 | |
| QVLMCode Generator=gpt-oss-120B, Segmentation Model=ConvNeXt, Protocol=Zero-shot, pass@1, Reasoning Config=Medium effort with 4096-token budget2026.01 | 43.84 | |
| QVLMCode Generator=GPT-5, Segmentation Model=DINOv3 + Mask2Former, Protocol=Zero-shot, pass@1, Reasoning Config=Minimal effort without token limits2026.01 | 40.74 | |
| QVLMCode Generator=Llama-3.1-8B, Segmentation Model=ConvNeXt, Protocol=Zero-shot, pass@12026.01 | 39.86 | |
| GPT-5Protocol=Zero-shot, pass@1, Input=Image+Text high-detail encoding2026.01 | 39.3 | |
| QWEN 30B A3B thinkingProtocol=Zero-shot, pass@1, Reasoning Config=4096-token reasoning budget2026.01 | 39.01 |