Embodied Question Answering on OpenEQA EM-EQA
86.8AccuracyHuman
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HumanFrame=Full2024.08 | 86.8 | — | |
| HumanFrame=Full2024.09 | 86.8 | — | |
| SpaceEra++Backbone=Qwen2.5-VL-7B2026.07 | 63.6 | 3.6 | |
| +SpaceAlignBackbone=Qwen2.5-VL-7B2026.07 | 61.1 | 3.6 | |
| +ScenePickBackbone=Qwen2.5-VL-7B2026.07 | 59.5 | 3.5 | |
| GPT-4o + Coarse CorrespondencesFrame=42024.08 | 59.1 | — | |
| SpaceEraBackbone=Qwen2.5-VL-7B2026.07 | 58.6 | 3.5 | |
| GPT-4V + Coarse CorrespondencesFrame=82024.08 | 58.5 | — | |
| GPT-4VFrame=502024.08 | 55.3 | — | |
| GPT-4VFrame=502024.09 | 55.3 | — | |
| GPT-4VFrame=152024.08 | 54.6 | — | |
| GPT-4VFrame=152024.09 | 54.6 | — | |
| LLaVA-3DFrame=322024.09 | 53.2 | — | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL-7B2026.07 | 50.1 | 3.1 | |
| GPT-4oFrame=42024.08 | 49.4 | — | |
| Gemini-ProFrame=152024.08 | 44.9 | — | |
| Gemini-ProFrame=152024.09 | 44.9 | — | |
| GPT-4VFrame=82024.08 | 44.8 | — | |
| Claude3Frame=202024.08 | 36.3 | — | |
| Claude3Frame=202024.09 | 36.3 | — | |
| GPT-4Frame=02024.08 | 33.5 | — | |
| GPT-4Frame=02024.09 | 33.5 | — | |
| LLaMA2Frame=02024.08 | 28.3 | — | |
| LLaMA2Frame=02024.09 | 28.3 | — |