Task Planning on EB-ALFRED Base
96Success Rate (SR)Claude-3.5-Sonnet
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Claude-3.5-SonnetType=Closed-source2025.08 | 96 | 97.5 | |
| Llama-3.2-90B-Vision-InsType=Open-source2025.08 | 94 | 94.5 | |
| Gemini-1.5-ProType=Closed-source2025.08 | 92 | 92.5 | |
| Claude-3.7-SonnetType=Closed-source2025.08 | 90 | — | |
| GPT-4oType=Closed-source2025.08 | 86 | 90.7 | |
| RoboMemoryType=Ours, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 86 | 88 | |
| InternVL3-78BType=Open-source2025.08 | 84 | — | |
| Gemini-2.0-flashType=Closed-source2025.08 | 82 | 82 | |
| InternVL2.5-78BType=Open-source2025.08 | 80 | 82 | |
| ReflexionType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 80 | 84.2 | |
| VoyagerType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 76 | 87 | |
| GPT-4o-miniType=Closed-source2025.08 | 74 | 77.5 | |
| Qwen2.5-VL-72B-InsType=Open-source2025.08 | 74 | — | |
| Claude-3.5-SonnetType=Closed-source2025.08 | 72 | 72 | |
| BrainMemEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 72 | 78.2 | |
| Gemini-1.5-ProType=Closed-source2025.08 | 70 | 74.3 | |
| Claude-3.7-SonnetType=Closed-source2025.08 | 68 | — | |
| RoboMemoryType=Ours, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 68 | 75.5 | |
| RoboMemoryEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 68 | 75.5 | |
| GPT-4oType=Closed-source2025.08 | 64 | 74 | |
| Gemini-2.0-flashType=Closed-source2025.08 | 62 | 65.7 | |
| CradleType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 62 | 67 | |
| InternVL2.5-38BType=Open-source2025.08 | 60 | 61.5 | |
| VoyagerType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 56 | 73.2 | |
| VoyagerEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 56 | 73.2 | |
| CradleType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 54 | 67.9 | |
| CradleEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 54 | 67.9 | |
| Qwen2.5-VL-72B-InsType=Open-source2025.08 | 50 | — | |
| ReflexionType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 48 | 54 | |
| ReflexionEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 48 | 54 | |
| Llama-3.2-90B-Vision-InsType=Open-source2025.08 | 38 | 43.7 | |
| InternVL2.5-78BType=Open-source2025.08 | 38 | 42.3 | |
| InternVL3-78BType=Open-source2025.08 | 38 | — | |
| RoboOSType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 38 | 47.8 | |
| InternVL2.5-38BType=Open-source2025.08 | 36 | 37.3 | |
| GPT-4o-miniType=Closed-source2025.08 | 34 | 47.8 | |
| RoboOSType=Baselines, Backbone=Qwen2.5-VL-72B-Ins2025.08 | 32 | 38.4 | |
| RoboOSType=Baselines, Backbone=RoboBrain2-32B2025.08 | 32 | 37.2 | |
| RoboOSEvaluation Model=Qwen2.5-VL-72B-Ins2026.03 | 32 | 38.4 | |
| RoboOSType=Baselines, Backbone=RoboBrain2-32B2025.08 | 28 | 34.8 |