Geospatial Question Answering on MapEval API
71.88Overall AccSpatial-Agent
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Spatial-AgentBackbone Model=GPT-5, Model Type=Closed-Source2026.01 | 71.88 | 85.94 | 53.01 | 75.76 | 77.61 | |
| Spatial-AgentBackbone Model=Qwen2.5-72B-Instruct, Model Type=Open-Source2026.01 | 53.41 | 68.75 | 39.76 | 43.94 | 61.19 | |
| Spatial-AgentBackbone Model=Qwen2.5-32B-Instruct, Model Type=Open-Source2026.01 | 52.35 | 71.88 | 39.76 | 50 | 47.76 | |
| Spatial-AgentBackbone Model=LlaMA-70B, Model Type=Open-Source2026.01 | 47.77 | 71.88 | 53.01 | 34.85 | 31.34 | |
| Spatial-AgentBackbone Model=GPT-4o-mini, Model Type=Closed-Source2026.01 | 45.15 | 70.31 | 33.73 | 30.3 | 46.27 | |
| ReflexionBackbone Model=GPT-4o-mini, Model Type=Closed-Source2026.01 | 38.29 | 65.63 | 22.89 | 30.3 | 34.33 | |
| MapEval APIBackbone Model=LlaMA-70B, Model Type=Open-Source2026.01 | 37.67 | 53.13 | 32.53 | 42.42 | 31.34 | |
| Spatial-AgentBackbone Model=GPT-3.5-Turbo, Model Type=Closed-Source2026.01 | 34.61 | 57.81 | 26.51 | 36.36 | 25.37 | |
| ReActBackbone Model=GPT-4o-mini, Model Type=Closed-Source2026.01 | 32.98 | 60.94 | 22.89 | 22.73 | 25.37 | |
| Direct LLMBackbone Model=GPT-4o-mini, Model Type=Closed-Source2026.01 | 32.23 | 45.31 | 32.53 | 22.73 | 28.36 | |
| MapEval APIBackbone Model=GPT-3.5-Turbo, Model Type=Closed-Source2026.01 | 27.33 | 39.06 | 22.89 | 33.33 | 19.4 | |
| MapEval APIBackbone Model=GPT-4o-mini, Model Type=Closed-Source2026.01 | 23 | 28.13 | 14.46 | 13.64 | 43.28 | |
| Spatial-AgentBackbone Model=Gemma-2-9B, Model Type=Open-Source2026.01 | 22.7 | 31.25 | 14.46 | 19.7 | 25.37 |