Embodied Question Answering on Episodic Memory EQA (EM-EQA)
86.8LLM MatchHuman baseline
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Human baseline2025.12 | 86.8 | 87.9 | 77.3 | 87.9 | 86.7 | 98.7 | 81.8 | 87.2 | — | |
| Human (Full trajectory)trajectory=Full2026.04 | 86.8 | — | — | — | — | — | — | — | — | |
| R42025.12 | 79.77 | 76.52 | 69.39 | 80.52 | 65.34 | 87.3 | 69.82 | 72.42 | — | |
| HGR2026.04 | 58.3 | — | — | — | — | — | — | — | 3.1 | |
| GPT-4V2025.12 | 55.3 | 51.4 | 53.3 | 65.2 | 42.6 | 57.7 | 63.8 | 52.3 | — | |
| Multi-Frame2026.04 | 48.1 | — | — | — | — | — | — | — | 3 | |
| Gemini 1.0 Pro Vision2025.12 | 44.9 | 41.5 | 33.3 | 41.9 | 37.6 | 56.9 | 52.2 | 52.1 | — | |
| GPT-4 w/ LLaVA-1.5visual_adapter=LLaVA-1.52025.12 | 43.6 | 36.5 | 31.9 | 45.8 | 36.1 | 56 | 54.8 | 44.8 | — | |
| GPT-4 w/ SVMrepresentation=Sparse Voxel Map2025.12 | 38.9 | 30 | 20 | 49.6 | 31.7 | 55.5 | 45.4 | 40.8 | — | |
| Frame Captions2026.04 | 38.1 | — | — | — | — | — | — | — | 0 | |
| LLaMA-2 w/ LLaVA-1.5visual_adapter=LLaVA-1.52025.12 | 36.8 | 30.5 | 18.8 | 39.4 | 31.4 | 50.1 | 47.4 | 41.7 | — | |
| GPT-4 w/ CGrepresentation=ConceptGraphs2025.12 | 36.5 | 26.4 | 17 | 40.6 | 29.1 | 55.5 | 48.4 | 39.9 | — | |
| Claude 32025.12 | 36.3 | 37 | 13.1 | 39.2 | 37 | 45.5 | 37.9 | 47.3 | — | |
| Blind LLM (GPT-4)2026.04 | 35.5 | — | — | — | — | — | — | — | 0 | |
| ConceptGraph Captions2026.04 | 34.4 | — | — | — | — | — | — | — | 0 | |
| LLaMA-2 w/ SVMrepresentation=Sparse Voxel Map2025.12 | 34.3 | 23.4 | 11.7 | 38.9 | 30.8 | 52.8 | 45.4 | 39.1 | — | |
| GPT-42025.12 | 33.5 | 15.4 | 20.3 | 31.5 | 31.4 | 51 | 52.2 | 34.2 | — | |
| LLaMA-2 w/ CGrepresentation=ConceptGraphs2025.12 | 28.7 | 17.1 | 13.9 | 24.4 | 27.2 | 43.5 | 38.1 | 39 | — | |
| LLaMA-22025.12 | 28.3 | 10.7 | 15.3 | 22.3 | 25 | 51.7 | 44.1 | 29.7 | — |