Embodied Task Completion on EB-Habitat
77Avg Success RateClaude-4.6
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude-4.6Tool Use=True2026.05 | 77 | 100 | 96 | 76 | 68 | 46 | 76 | — | — | — | — | — | — | — | |
| Claude-3.7Tool Use=True2026.05 | 76 | 100 | 94 | 82 | 66 | 43 | 70 | — | — | — | — | — | — | — | |
| GPT-5Tool Use=True2026.05 | 75 | 98 | 82 | 72 | 76 | 46 | 74 | — | — | — | — | — | — | — | |
| Claude-3.5-SonnetModel Type=Proprietary MLLM, Enhancement=BrainMem2026.03 | 73 | 96 | 74 | 84 | 46 | 64 | 74 | — | — | — | — | — | — | — | |
| Claude-3.5-SonnetModel Type=Proprietary MLLM, Enhancement=None2026.03 | 68 | 96 | 68 | 78 | 38 | 58 | 70 | — | — | — | — | — | — | — | |
| Claude-3.7Tool Use=False2026.05 | 67 | 98 | 78 | 76 | 40 | 43 | 68 | — | — | — | — | — | — | — | |
| Claude-4.6Tool Use=False2026.05 | 67 | 98 | 72 | 70 | 40 | 46 | 74 | — | — | — | — | — | — | — | |
| Gemini-1.5-proModel Type=Proprietary MLLM, Enhancement=BrainMem2026.03 | 64.3 | 94 | 60 | 58 | 46 | 60 | 68 | — | — | — | — | — | — | — | |
| GPT-4oTool Use=True2026.05 | 64 | 96 | 72 | 60 | 62 | 40 | 52 | — | — | — | — | — | — | — | |
| GPT-4oModel Type=Proprietary MLLM, Enhancement=BrainMem2026.03 | 63.3 | 90 | 46 | 60 | 42 | 68 | 74 | — | — | — | — | — | — | — | |
| ReActBackbone=Claude-3.7-Sonnet2026.01 | 61.2 | 90 | 58 | 58 | 38 | — | 62 | 70.8 | 97.5 | 68.5 | 79.5 | 72 | 43.8 | — | |
| GPT-4oModel Type=Proprietary MLLM, Enhancement=None2026.03 | 59 | 86 | 44 | 56 | 36 | 64 | 68 | — | — | — | — | — | — | — | |
| Claude-3.7-SonnetModel Type=Proprietary MLLM, Enhancement=None2026.03 | 58.7 | 90 | 58 | 58 | 38 | 46 | 62 | — | — | — | — | — | — | — | |
| ReActBackbone=GPT-4o2026.01 | 58 | 86 | 44 | 56 | 36 | — | 68 | 70.7 | 90.7 | 56 | 68 | 75.2 | 62.1 | — | |
| ReActBackbone=Gemini-1.5-Pro2026.01 | 57.2 | 92 | 52 | 48 | 38 | — | 56 | 61 | 92.5 | 53.5 | 49.5 | 59.4 | 50 | — | |
| GPT-5Tool Use=False2026.05 | 57 | 84 | 42 | 64 | 40 | 46 | 66 | — | — | — | — | — | — | — | |
| Gemini-1.5-ProModel Type=Proprietary MLLM, Enhancement=None2026.03 | 56.3 | 92 | 52 | 48 | 38 | 52 | 56 | — | — | — | — | — | — | — | |
| InternVL3-78BModel Type=Open-Source MLLM, Enhancement=None2026.03 | 55 | 84 | 58 | 60 | 32 | 40 | 56 | — | — | — | — | — | — | — | |
| WorldMindBackbone=GPT-4.1-mini2026.01 | 50.8 | 86 | 44 | 44 | 34 | — | 46 | 57.2 | 87 | 48.5 | 51.5 | 52.3 | 46.7 | — | |
| InternVL2_5-78BModel Type=Open-Source MLLM, Enhancement=None2026.03 | 49 | 80 | 42 | 56 | 30 | 28 | 58 | — | — | — | — | — | — | — | |
| Qwen2.5-32BTool Use=True2026.05 | 49 | 90 | 46 | 58 | 56 | 18 | 28 | — | — | — | — | — | — | — | |
| WorldMindBackbone=GPT-3.5-turbo2026.01 | 48.8 | 86 | 42 | 44 | 32 | — | 40 | 56.7 | 89 | 47 | 51 | 49.8 | 46.8 | — | |
| SimuRABackbone=GPT-4.1-mini2026.01 | 48.4 | 84 | 42 | 40 | 32 | — | 44 | 55.4 | 85 | 45.5 | 47 | 48.2 | 51.5 | — | |
| SimuRABackbone=GPT-3.5-turbo2026.01 | 48 | 82 | 40 | 36 | 38 | — | 44 | 55.3 | 83 | 47 | 40 | 48.9 | 57.8 | — | |
| GPT-4oTool Use=False2026.05 | 48 | 92 | 32 | 50 | 30 | 40 | 46 | — | — | — | — | — | — | — | |
| ReActBackbone=InternVL2.5-78B2026.01 | 47.6 | 84 | 30 | 58 | 32 | — | 34 | 55.2 | 82 | 43 | 59 | 63.9 | 45.1 | — | |
| SynapseBackbone=GPT-3.5-turbo2026.01 | 47.2 | 84 | 40 | 38 | 38 | — | 42 | 54.2 | 85 | 43 | 43 | 46.3 | 53.5 | — | |
| ReasoningBankBackbone=GPT-3.5-turbo2026.01 | 46.4 | 76 | 40 | 40 | 34 | — | 42 | 53.9 | 78.5 | 41.5 | 44 | 46.2 | 59.5 | — | |
| AWMBackbone=GPT-3.5-turbo2026.01 | 46.4 | 78 | 42 | 44 | 32 | — | 36 | 50.9 | 78 | 43 | 48 | 39.7 | 45.7 | — | |
| ReasoningBankBackbone=GPT-4.1-mini2026.01 | 46.4 | 76 | 42 | 30 | 32 | — | 52 | 53.7 | 77 | 45.5 | 34.5 | 58.4 | 53.3 | — | |
| ReActBackbone=Llama-3.2-90B-Vis2026.01 | 45.6 | 94 | 24 | 50 | 28 | — | 32 | 50.6 | 94.5 | 32.5 | 53 | 39.7 | 59.6 | — | |
| SynapseBackbone=GPT-4.1-mini2026.01 | 45.6 | 78 | 42 | 38 | 36 | — | 34 | 52 | 78.3 | 46.5 | 45.5 | 39.7 | 50 | — | |
| Qwen-VL-MaxModel Type=Proprietary MLLM, Enhancement=None2026.03 | 45.3 | 74 | 40 | 50 | 30 | 36 | 42 | — | — | — | — | — | — | — | |
| BoNBackbone=GPT-4.1-mini2026.01 | 44.4 | 74 | 32 | 36 | 34 | — | 46 | 50.9 | 75.8 | 38 | 41.5 | 48.9 | 50.5 | — | |
| Qwen3-32BTool Use=True2026.05 | 44 | 88 | 46 | 46 | 66 | 6 | 14 | — | — | — | — | — | — | — | |
| ReActBackbone=GPT-3.5-turbo2026.01 | 43.6 | 82 | 30 | 34 | 34 | — | 38 | 50.4 | 84 | 34 | 40 | 43.4 | 50.4 | — | |
| BoNBackbone=GPT-3.5-turbo2026.01 | 43.6 | 74 | 34 | 36 | 34 | — | 40 | 50.8 | 75.5 | 35.5 | 43 | 46.9 | 53.3 | — | |
| AWMBackbone=GPT-4.1-mini2026.01 | 43.6 | 74 | 32 | 38 | 30 | — | 44 | 50.8 | 74.8 | 37 | 44.5 | 47.9 | 49.7 | — | |
| ReActBackbone=GPT-4.1-mini2026.01 | 41.6 | 74 | 26 | 32 | 36 | — | 40 | 47.4 | 76 | 30 | 37 | 43.7 | 50.3 | — | |
| Llama-3.2-90B-VisionModel Type=Open-Source MLLM, Enhancement=None2026.03 | 40.3 | 94 | 24 | 50 | 28 | 14 | 32 | — | — | — | — | — | — | — | |
| Gemini-1.5-flashModel Type=Proprietary MLLM, Enhancement=None2026.03 | 39.3 | 76 | 32 | 48 | 32 | 12 | 36 | — | — | — | — | — | — | — | |
| Qwen3-8BTool Use=True2026.05 | 39 | 74 | 38 | 38 | 46 | 12 | 24 | — | — | — | — | — | — | — | |
| Qwen2.5-VL-72BModel Type=Open-Source MLLM, Enhancement=None2026.03 | 37.7 | 74 | 28 | 42 | 24 | 18 | 40 | — | — | — | — | — | — | — | |
| Ovis2-34BModel Type=Open-Source MLLM, Enhancement=None2026.03 | 37 | 68 | 34 | 38 | 30 | 14 | 38 | — | — | — | — | — | — | — | |
| ReActBackbone=GPT-4o-mini2026.01 | 36.4 | 74 | 22 | 32 | 32 | — | 22 | 44 | 77.5 | 32.5 | 42 | 33.1 | 57.8 | — | |
| Qwen2.5-32BTool Use=False2026.05 | 34 | 68 | 26 | 48 | 20 | 18 | 26 | — | — | — | — | — | — | — | |
| Gemma-3-12B-IT + µOnline RLModel Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=µOnline RL2026.01 | 33.8 | 60 | 13 | 37 | 35 | 24 | — | — | — | — | — | — | — | — | |
| GPT-4o-miniModel Type=Proprietary MLLM, Enhancement=None2026.03 | 32.7 | 74 | 22 | 32 | 32 | 14 | 22 | — | — | — | — | — | — | — | |
| Gemma-3-12B-IT + µOffline Sup.Model Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=µOffline Sup.2026.01 | 31.8 | 66 | 14 | 35 | 27 | 17 | — | — | — | — | — | — | — | — | |
| Gemma-3-12B-IT + µSimpleModel Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=µSimple2026.01 | 31.2 | 62 | 15 | 30 | 31 | 18 | — | — | — | — | — | — | — | — | |
| Qwen3-32BTool Use=False2026.05 | 30 | 78 | 22 | 30 | 36 | 6 | 10 | — | — | — | — | — | — | — | |
| Qwen3.5-35BTool Use=True2026.05 | 26 | 52 | 22 | 26 | 46 | 2 | 10 | — | — | — | — | — | — | — | |
| Gemma-3-12B-ITModel Backbone=Gemma-3-12B-IT, Memory Head Variation (MemCtrl)=None2026.01 | 23 | 58 | 10 | 24 | 24 | 4 | — | — | — | — | — | — | — | — | |
| Qwen3-8BTool Use=False2026.05 | 23 | 44 | 14 | 24 | 18 | 12 | 24 | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7B-Ins + µOffline Sup.Model Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=µOffline Sup.2026.01 | 22.8 | 39 | 5 | 33 | 17 | 20 | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7B-Ins + µOnline RLModel Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=µOnline RL2026.01 | 22.2 | 37 | 3 | 37 | 16 | 18 | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7B-Ins + µSimpleModel Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=µSimple2026.01 | 21.4 | 39 | 10 | 31 | 14 | 13 | — | — | — | — | — | — | — | — | |
| Argos2025.12 | 20.7 | 45.3 | 12 | 24 | 17.3 | 9.3 | 16 | — | — | — | — | — | — | — | |
| Video-R12025.12 | 16.2 | 46.7 | 6 | 15.3 | 16.7 | 3.3 | 9.3 | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7B-InsModel Backbone=Qwen2.5-VL-7B-Ins, Memory Head Variation (MemCtrl)=None2026.01 | 14.3 | 32 | 2 | 26 | 14 | 2 | — | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7Bprompting=Chain-of-Thought (CoT)2025.12 | 12.6 | 30.7 | 6.7 | 13.3 | 12 | 1.3 | 11.3 | — | — | — | — | — | — | — | |
| Qwen3.5-35BTool Use=False2026.05 | 11 | 30 | 6 | 12 | 10 | 2 | 6 | — | — | — | — | — | — | — | |
| Qwen2.5-VL-7Bprompting=base2025.12 | 9 | 23.3 | 5.3 | 10.7 | 8 | 0 | 6.7 | — | — | — | — | — | — | — | |
| ACE-Brain-0-8BAccess=Embodied Brain MLLMs, Parameters=8B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 42.3 | |
| GPT-4oAccess=Closed-source2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 59 | |
| InternVL3-8BAccess=Open-source general-purpose, Parameters=8B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.3 | |
| InternVL3.5-8BAccess=Open-source general-purpose, Parameters=8B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 32 | |
| MiMo-Embodied-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 16.7 | |
| Pelican-VL-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 16.3 | |
| Qwen-VL-MaxAccess=Closed-source2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 45.3 | |
| Qwen2.5-VL-7B-Inst.Access=Open-source general-purpose, Parameters=7B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 14.3 | |
| Qwen3-VL-8B-Inst.Access=Open-source general-purpose, Parameters=8B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.7 | |
| RoboBrain2.0-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.3 | |
| RoboBrain2.5-8BAccess=Embodied Brain MLLMs, Parameters=8B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.3 | |
| VeBrain-7BAccess=Embodied Brain MLLMs, Parameters=7B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 15 | |
| Vlaser-8BAccess=Embodied Brain MLLMs, Parameters=8B2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | 40 |