Embodied Question Answering on A-EQA
85.1Overall (LLM-Match)Human Agent
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Human AgentModel=Human2026.02 | 85.1 | 89.7 | — | 72.8 | — | 85.4 | — | 84.8 | — | 97.8 | — | 78.9 | — | 88.5 | — | — | — | — | |
| OursExploration Strategy=Commercial MLLM Exploration, Model=GPT-4o2026.02 | 65.6 | 62 | 52.7 | 63.7 | 48.6 | 68.5 | 52.5 | 52.2 | 36.8 | 91.2 | 54.1 | 66.4 | 45.5 | 54.5 | 44.3 | 48.7 | — | — | |
| ReEXploreExploration Strategy=Commercial MLLM Exploration, Model=GPT-4o2026.02 | 58.3 | 37.5 | 21.9 | 65.6 | 45.8 | 67.9 | 46.1 | 60 | 25.3 | 100 | 53.5 | 65 | 42.4 | 35 | 23.6 | 37.3 | — | — | |
| GSMemCategory=VLM Exploration, VLM Backbone=GPT-4o2026.03 | 55.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 43.8 | — | — | |
| ObsGraph2026.06 | 54.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 45 | — | — | |
| 3D-MemExploration Strategy=Commercial MLLM Exploration, Model=GPT-4o2026.02 | 54.4 | 35 | 18.8 | 50 | 37.3 | 64.3 | 56.3 | 50 | 24.7 | 80 | 49.3 | 50 | 22.1 | 30 | 21.4 | 33.3 | — | — | |
| 3D-MemCategory=VLM Exploration, VLM Backbone=GPT-4o2026.03 | 52.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 42 | — | — | |
| 3D-Mem†2026.06 | 49.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37 | — | — | |
| GPT-4oCategory=VLMs2026.03 | 47.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CG + FrontierExploration Strategy=Commercial MLLM Exploration, Model=GPT-4o2026.02 | 47.2 | 45 | 42 | 32.1 | 25 | 50.8 | 35.2 | 32.9 | 18.7 | 68.5 | 38.4 | 58.8 | 42.2 | 45.5 | 33.5 | 33.3 | — | — | |
| CG w/ Frontier SnapshotsCategory=VLM Exploration, VLM Backbone=GPT-4o2026.03 | 47.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 33.3 | — | — | |
| Explore-EQAExploration Strategy=Commercial MLLM Exploration, Model=GPT-4o2026.02 | 46.9 | 44 | 19.6 | 37.1 | 29.6 | 55.3 | 36 | 42.1 | 6.6 | 46.3 | 9.2 | 63.2 | 35.7 | 45.5 | 22 | 23.4 | — | — | |
| Explore-EQACategory=VLM Exploration, VLM Backbone=GPT-4o2026.03 | 46.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.4 | — | — | |
| Explore-EQA2026.06 | 46.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.4 | — | — | |
| ReEXploreExploration Strategy=Open-Sourced MLLM-based Exploration, Model=Qwen2.5-VL2026.02 | 46.2 | 50.6 | 31.3 | 29.4 | 17.8 | 49.1 | 31 | 43.1 | 15.1 | 66.7 | 17.2 | 47.9 | 27.4 | 43.1 | 20.9 | 23 | — | — | |
| Qwen3-VL-8B-InstructCategory=VLMs2026.03 | 44 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| OursExploration Strategy=Open-Sourced MLLM-based Exploration, Model=Qwen2.5-VL2026.02 | 43.1 | 39 | 29.5 | 28.4 | 17.3 | 51.3 | 32.4 | 37.6 | 14.4 | 58.3 | 24.4 | 48.5 | 24.8 | 47.2 | 22.3 | 23.7 | — | — | |
| Multi-FrameExploration Strategy=Socratic LLM-based Exploration w/ Scene-Graph Captions, Model=Multi-Frame2026.02 | 41.8 | 34 | — | 34.3 | — | 51.5 | — | 39.5 | — | 51.9 | — | 45.6 | — | 36.6 | — | — | — | — | |
| Multi-Frame2026.06 | 41.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7.5 | — | — | |
| 3D-MemExploration Strategy=Open-Sourced MLLM-based Exploration, Model=Qwen2.5-VL2026.02 | 39.1 | 25 | 13.9 | 23.8 | 7.7 | 52.6 | 29.3 | 33.3 | 3.9 | 57.5 | 26.1 | 43.8 | 13.5 | 37.5 | 8.4 | 14.6 | — | — | |
| LLaVA-1.5Exploration Strategy=Socratic LLM-based Exploration w/ Scene-Graph Captions, Model=LLaVA-1.52026.02 | 38.1 | 25 | — | 24 | — | 34.1 | — | 34.4 | — | 56.9 | — | 53.5 | — | 40.6 | — | — | — | — | |
| Frame Cap.2026.06 | 38.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 7 | — | — | |
| TANGO2026.06 | 37.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4oExploration Strategy=Socratic LLM-based Exploration w/ Frame Captions, Model=GPT-4o2026.02 | 35.9 | 22 | — | 25 | — | 27.3 | — | 40.8 | — | 50.9 | — | 61.8 | — | 38.4 | — | — | — | — | |
| GPT-4oCategory=Blind LLMs2026.03 | 35.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4Exploration Strategy=Socratic LLM-based Exploration w/ Frame Captions, Model=GPT-42026.02 | 35.5 | 25.3 | — | 28.4 | — | 27.3 | — | 37.7 | — | 47.2 | — | 54.2 | — | 29.5 | — | — | — | — | |
| CG Scene-GraphExploration Strategy=Socratic LLM-based Exploration w/ Scene-Graph Captions, Model=CG Scene-Graph2026.02 | 34.4 | 25.3 | — | 16.5 | — | 29.2 | — | 37 | — | 52.2 | — | 46.8 | — | 37.8 | — | — | — | — | |
| CG Cap.2026.06 | 34.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.5 | — | — | |
| SVM Scene-GraphExploration Strategy=Socratic LLM-based Exploration w/ Scene-Graph Captions, Model=SVM Scene-Graph2026.02 | 34.2 | 29 | — | 17.2 | — | 31.5 | — | 31.5 | — | 54.2 | — | 39.8 | — | 38.9 | — | — | — | — | |
| SVM Cap.2026.06 | 34.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 6.4 | — | — | |
| GPT-4oCategory=LLM with Frame Captions, Captions=LLaVA-1.5-13B2026.03 | 33.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA-3.1-8B-InstructCategory=Blind LLMs2026.03 | 32.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA-3.1Category=LLM with Frame Captions, Captions=LLaVA-1.5-13B2026.03 | 30.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| 3D-MemFR=true, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 52.6 | 42 | |
| CG + FrontierFR=true, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 47.2 | 33.3 | |
| CG Scene-Graph CaptionsFR=false, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.4 | 6.5 | |
| Explore-EQAFR=true, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 46.9 | 23.4 | |
| LLaVA-1.5 Frame CaptionsFR=false, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 38.1 | 7 | |
| MTU3DFR=true, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 51.1 | 42.6 | |
| Multi-FrameFR=false, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 41.8 | 7.5 | |
| SCOPEFR=true, FI=true2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 59.1 | 41 | |
| SVM Scene-Graph CaptionsFR=false, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.2 | 6.4 | |
| TANGOFR=false, FI=false2025.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.2 | — |