List your company

ResearchFormerly WizWand

OpenEQA

Benchmarks

TaskDatasetSOTA resultTrendResults

Embodied Question Answering

OpenEQA v1.0 (test)

67.7LLM-Match

32

Episodic-memory Question Answering

OpenEQA v1 (ScanNet)

87.7LLM-Match

29

Embodied Question Answering

OpenEQA

69.3Accuracy

26

Embodied Question Answering

OpenEQA EM-EQA

86.8Accuracy

24

Open-ended Video Reasoning

OpenEQA (test)

58.6Accuracy (< 30s)

18

3D Question Answering

OpenEQA

56.2LLM-Match

12

Active Question Answering

OpenEQA v1 (HM3D)

85.1LLM-Match

12

Episodic-memory Question Answering

OpenEQA v1 (HM3D)

85.1LLM Match Score

12

Embodied Question Answering

OpenEQA EM-EQA Episodes up to 32 frames

86.8LLM-Match Score

10

Zero-shot understanding of embodied scenes

OpenEQA

51.2Score

10

Embodied Question Answering

OpenEQA (test)

86.8GPT-4 Score

10

Embodied Question Answering

OpenEQA EM-EQA

86.8LLM-Match

8

Episodic Memory Embodied Question Answering

OpenEQA 1/10 subset Unseen (val)

61.5LLM Match Score

7

Embodied Question Answering

OpenEQA v1 (test)

85.1Score

5

Active Embodied Question Answering

OpenEQA 184

52.6LLM-SR

4