Embodied Question Answering on OpenEQA
69.3AccuracyGPT-5.4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-5.4Model Category=General-Purpose VLMs2026.06 | 69.3 | — | |
| Gemini-2.5-ProModel Category=General-Purpose VLMs2026.06 | 65.8 | — | |
| Qwen3-VL-8BModel Category=General-Purpose VLMs2026.06 | 65.1 | — | |
| GPT-5Access Mode=API call only2026.05 | 64.4 | — | |
| Embodied-R1.5Model Category=Ours2026.06 | 62.6 | — | |
| GPT-5-miniAccess Mode=API call only2026.05 | 59.2 | — | |
| GR-ER 1.5 ThinkingAccess Mode=API call only2026.05 | 55 | — | |
| Gemini 2.5 ProAccess Mode=API call only2026.05 | 55 | — | |
| Pelican-VL-1.0Model Category=Embodied VLMs2026.06 | 54.7 | — | |
| Mimo-EmbodiedModel Category=Embodied VLMs2026.06 | 53.6 | — | |
| GPT-4oModel Category=General-Purpose VLMs2026.06 | 51.1 | — | |
| GR-ER 1.5Access Mode=API call only2026.05 | 50.5 | — | |
| Gemini-Robotics-ER-1.5Model Category=Embodied VLMs2026.06 | 50.5 | — | |
| Qwen3-VL-8BAccess Mode=Open weights only2026.05 | 49.3 | — | |
| Qwen3-VL-4BAccess Mode=Open weights only2026.05 | 47.6 | — | |
| Molmo2Access Mode=Open weights, Open data, Open code2026.05 | 46.9 | — | |
| Molmo2-ERAccess Mode=Open weights, Open data, Open code2026.05 | 44.7 | — | |
| InternVL3.5-4BAccess Mode=Open weights only2026.05 | 44.6 | — | |
| InternVL3.5-8BAccess Mode=Open weights only2026.05 | 43.5 | — | |
| InternVL3.5-8BModel Category=General-Purpose VLMs2026.06 | 42.9 | — | |
| LLaVA-OV-7BAccess Mode=Open weights only2026.05 | 36.2 | — | |
| Molmo-D-7BModel Category=General-Purpose VLMs2026.06 | 35.1 | — | |
| Embodied-R1Model Category=Embodied VLMs2026.06 | 34.5 | — | |
| MagmaModel Category=Embodied VLMs2026.06 | 32.7 | — | |
| VeBrainModel Category=Embodied VLMs2026.06 | 27.8 | — | |
| RoboBrain-2.0-7BModel Category=Embodied VLMs2026.06 | 24.2 | — | |
| ACE-Brain-0-8BAccess=Embodied Brain MLLMs, Parameters=8B2026.03 | — | 70 | |
| Fast-ThinkAct-7BModel Size=7B2026.01 | — | 59 | |
| GPT-4oAccess=Closed-source2026.03 | — | 56.4 | |
| InternVL2.5-8BModel Size=8B2026.01 | — | 54.4 | |
| InternVL3-8BModel Size=8B2026.01 | — | 55.5 | |
| InternVL3-8BAccess=Open-source general-purpose, Parameters=8B2026.03 | — | 68.3 | |
| InternVL3.5-8BAccess=Open-source general-purpose, Parameters=8B2026.03 | — | 64.9 | |
| Magma-8BModel Size=8B2026.01 | — | 49.1 | |
| MiMo-Embodied-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | 74.1 | |
| NVILA-8BModel Size=8B2026.01 | — | 54 | |
| Pelican-VL-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | 63.3 | |
| Qwen2.5-VL-7BModel Size=7B2026.01 | — | 50.8 | |
| Qwen2.5-VL-7B-Inst.Access=Open-source general-purpose, Parameters=7B2026.03 | — | 59.9 | |
| Qwen3-VL-2B-Inst.Access=Open-source general-purpose, Parameters=2B2026.03 | — | 56.4 | |
| Qwen3-VL-8B-Inst.Access=Open-source general-purpose, Parameters=8B2026.03 | — | 67.1 | |
| RoboBrain2.0-7BModel Size=7B2026.01 | — | 51.1 | |
| RoboBrain2.0-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | 60 | |
| RoboBrain2.5-8BAccess=Embodied Brain MLLMs, Parameters=8B2026.03 | — | 62.6 | |
| ThinkAct-7BModel Size=7B2026.01 | — | 56.2 | |
| VeBrain-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | 63.8 | |
| Vlaser-8BAccess=Embodied Brain MLLMs, Parameters=8B2026.03 | — | 56.3 |