Embodied reasoning on ERQA (test)
70.25AccuracyR4
Evaluation results
| Method | Links | |
|---|---|---|
| R4Backbone=Gemma3-4B-IT, Vision Backbone=SAM2 Hiera Large, Backend=MapAnything2025.12 | 70.25 | |
| GPT-52025.12 | 65.7 | |
| o32025.12 | 64 | |
| Qwen3 VL 235B A22B ThinkingModel Size=235B A22B2025.12 | 52.5 | |
| Qwen3 VL 32B ThinkingModel Size=32B2025.12 | 52.3 | |
| Gemini 2.0 Pro Experimental2025.12 | 48.3 | |
| GPT-4o2025.12 | 47 | |
| Gemini 2.0 Flash2025.12 | 46.3 | |
| Gemini 1.5 Flash2025.12 | 42.3 | |
| Gemini 1.5 Pro2025.12 | 41.8 | |
| GPT-4o-mini2025.12 | 37.3 | |
| Claude 3.5 Sonnet2025.12 | 35.5 |