Embodied Question Answering on OpenEQA (test)
86.8GPT-4 ScoreHuman
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Human2025.01 | 86.8 | — | |
| Human2025.01 | 86.8 | — | |
| Tarsier22025.01 | 58.7 | — | |
| Tarsier22025.01 | 58.7 | — | |
| GPT-4V2025.01 | 55.3 | — | |
| GPT-4V2025.01 | 55.3 | — | |
| MLCD2025.01 | 48.8 | — | |
| MLCD2025.01 | 48.8 | — | |
| Gemini-1.5-Pro2025.01 | 44.9 | — | |
| Gemini-1.5-Pro2025.01 | 44.9 | — | |
| Claude 3evaluation_mode=Zero-shot2025.05 | — | 36.3 | |
| Gemini-Proevaluation_mode=Zero-shot2025.05 | — | 44.9 | |
| GPT-4evaluation_mode=Zero-shot2025.05 | — | 33.5 | |
| GPT-4Vevaluation_mode=Zero-shot, Number of input frames=50 frames2025.05 | — | 55.3 | |
| GPT-4Vevaluation_mode=Zero-shot, Number of input frames=15 frames2025.05 | — | 54.6 | |
| Humanevaluation_mode=Zero-shot2025.05 | — | 86.8 | |
| LLaMA2evaluation_mode=Zero-shot2025.05 | — | 28.3 | |
| VLM-3Revaluation_mode=Zero-shot2025.05 | — | 61.7 |