Task Planning on EB-Habitat Long
64Success Rate (SR)GPT-4o
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT-4oType=Closed-source2025.08 | 64 | 72.2 | |
| RoboMemoryType=Ours, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 62 | 74 | |
| RoboMemoryEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 62 | 74 | |
| BrainMemEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 62 | 75.3 | |
| Claude-3.5-SonnetType=Closed-source2025.08 | 58 | 63.3 | |
| Gemini-1.5-ProType=Closed-source2025.08 | 52 | 61.2 | |
| Claude-3.7-SonnetType=Closed-source2025.08 | 46 | — | |
| InternVL3-78BType=Open-source2025.08 | 40 | — | |
| CradleType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 30 | 52.1 | |
| CradleEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 30 | 52.1 | |
| InternVL2.5-78BType=Open-source2025.08 | 28 | 38.2 | |
| InternVL2.5-38BType=Open-source2025.08 | 28 | 35 | |
| Gemini-2.0-flashType=Closed-source2025.08 | 26 | 36.2 | |
| VoyagerType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 22 | 51 | |
| VoyagerEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 22 | 51 | |
| RoboOSType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 20 | 28.2 | |
| RoboOSEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 20 | 28.2 | |
| Qwen2.5-VL-72B-InsType=Open-source2025.08 | 18 | — | |
| ReflexionType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 15 | 33 | |
| ReflexionEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 15 | 33 | |
| GPT-4o-miniType=Closed-source2025.08 | 14 | 21.3 | |
| Llama-3.2-90B-Vision-InsType=Open-source2025.08 | 14 | 24.3 | |
| RoboOSType=Baselines, Backbone=RoboBrain2-32B2025.08 | 12 | 17.4 |