Episodic-memory Question Answering on OpenEQA v1 (ScanNet)
87.7LLM-MatchHuman baseline
Evaluation results
| Method | Links | |
|---|---|---|
| Human baseline2025.12 | 87.7 | |
| Human baseline2025.12 | 86.8 | |
| R42025.12 | 81.22 | |
| R42025.12 | 79.77 | |
| GPT-52025.12 | 64.4 | |
| GPT-5-mini2025.12 | 59.2 | |
| 3D-Mem2025.12 | 57.2 | |
| GR-ER1.5 w/ ThinkingMode=Thinking2025.12 | 55 | |
| GPT-4V2025.12 | 51.3 | |
| GR-ER1.52025.12 | 50.5 | |
| GPT-4V2025.12 | 49.6 | |
| AlanaVLM2025.12 | 47.8 | |
| AlanaVLM2025.12 | 46.7 | |
| GPT-4 w/ LLaVA-1.5Vision-Language Model=LLaVA-1.52025.12 | 45.4 | |
| GPT-4 w/ LLaVA-1.5Vision-Language Model=LLaVA-1.52025.12 | 43.6 | |
| GPT-4 w/ SVMRepresentation=Sparse Voxel Map2025.12 | 40.9 | |
| LLaMA-2 w/ LLaVA-1.5Vision-Language Model=LLaVA-1.52025.12 | 39.6 | |
| GPT-4 w/ SVMRepresentation=Sparse Voxel Map2025.12 | 38.9 | |
| GPT-4 w/ CGRepresentation=Concept Graphs2025.12 | 37.8 | |
| LLaMA-2 w/ LLaVA-1.5Vision-Language Model=LLaVA-1.52025.12 | 36.8 | |
| GPT-4 w/ CGRepresentation=Concept Graphs2025.12 | 36.5 | |
| LLaMA-2 w/ SVMRepresentation=Sparse Voxel Map2025.12 | 36 | |
| LLaMA-2 w/ SVMRepresentation=Sparse Voxel Map2025.12 | 34.3 | |
| GPT-42025.12 | 33.5 | |
| GPT-42025.12 | 32.5 | |
| LLaMA-2 w/ CGRepresentation=Concept Graphs2025.12 | 31 | |
| LLaMA-2 w/ CGRepresentation=Concept Graphs2025.12 | 28.7 | |
| LLaMA-22025.12 | 28.3 | |
| LLaMA-22025.12 | 27.9 |