Episodic-memory Question Answering on OpenEQA v1 (HM3D)
85.1LLM Match ScoreHuman baseline
Evaluation results
| Method | Links | |
|---|---|---|
| Human baseline2025.12 | 85.1 | |
| R42025.12 | 76.96 | |
| GPT-4V2025.12 | 46.6 | |
| AlanaVLM2025.12 | 44.8 | |
| GPT-4 w/ LLaVA-1.5Vision-Language Model=LLaVA-1.52025.12 | 40 | |
| GPT-42025.12 | 35.5 | |
| GPT-4 w/ SVMRepresentation=Sparse Voxel Map2025.12 | 35 | |
| GPT-4 w/ CGRepresentation=Concept Graphs2025.12 | 34 | |
| LLaMA-2 w/ LLaVA-1.5Vision-Language Model=LLaVA-1.52025.12 | 31.1 | |
| LLaMA-2 w/ SVMRepresentation=Sparse Voxel Map2025.12 | 30.9 | |
| LLaMA-22025.12 | 29 | |
| LLaMA-2 w/ CGRepresentation=Concept Graphs2025.12 | 24.2 |