Embodied Task Execution on ROBOTHOR
40Success Rate (SR)ReAct
Evaluation Results
| Method | Links | |
|---|---|---|
| ReActMemory=Base, Reflection=Reflexion, Backbone=GPT-4o mini2026.06 | 40 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=GPT-4o mini2026.06 | 30 | |
| ReActMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen2.5-9B2026.06 | 30 | |
| ReActMemory=Base, Reflection=None, Backbone=GPT-4o mini2026.06 | 20 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=GPT-4o mini2026.06 | 20 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 20 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 20 | |
| CoTMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| CoTMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| Plan&SolveMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 20 | |
| Plan&SolveMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 20 | |
| Plan&SolveMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| Plan&SolveMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 20 | |
| MADMemory=Base, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| MADMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 20 | |
| MADMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 20 | |
| MADMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 20 | |
| ReActMemory=Base, Reflection=Self-Refine, Backbone=GPT-4o mini2026.06 | 20 | |
| ReActMemory=Base, Reflection=Reflexion, Backbone=Qwen2.5-9B2026.06 | 20 | |
| ReActMemory=DC, Reflection=Reflexion, Backbone=Qwen2.5-9B2026.06 | 20 | |
| ReActMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen2.5-27B2026.06 | 20 | |
| MADMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen2.5-27B2026.06 | 20 | |
| NoneMemory=Base, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 10 | |
| NoneMemory=Base, Reflection=None, Backbone=GPT-4o mini2026.06 | 10 | |
| ReActMemory=Base, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 10 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| ReActMemory=DC, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 10 | |
| ReActMemory=DC, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 10 | |
| ReActMemory=DC, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| ReActMemory=DC, Reflection=None, Backbone=GPT-4o mini2026.06 | 10 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 10 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 10 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=GPT-4o mini2026.06 | 10 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 10 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=GPT-4o mini2026.06 | 10 | |
| CoTMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| CoTMemory=DC, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| CoTMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 10 | |
| CoTMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 10 | |
| CoTMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| CoTMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 10 | |
| CoTMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 10 | |
| CoTMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=None, Backbone=GPT-4o mini2026.06 | 10 | |
| Plan&SolveMemory=DC, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 10 | |
| Plan&SolveMemory=DC, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| Plan&SolveMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| MADMemory=Base, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 10 | |
| MADMemory=Base, Reflection=None, Backbone=GPT-4o mini2026.06 | 10 | |
| MADMemory=DC, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| MADMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| ReActMemory=Base, Reflection=Self-Refine, Backbone=Qwen2.5-27B2026.06 | 10 | |
| ReActMemory=Base, Reflection=Self-Refine, Backbone=Qwen2.5-9B2026.06 | 10 | |
| ReActMemory=Base, Reflection=Self-Refine, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| CoTMemory=Base, Reflection=Self-Refine, Backbone=Qwen2.5-27B2026.06 | 10 | |
| CoTMemory=SimpleMem, Reflection=Self-Refine, Backbone=Qwen2.5-9B2026.06 | 10 | |
| CoTMemory=SimpleMem, Reflection=Self-Refine, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=Self-Refine, Backbone=Qwen2.5-27B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=Self-Refine, Backbone=Qwen2.5-9B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=Self-Refine, Backbone=Qwen2.5-27B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=Self-Refine, Backbone=Qwen2.5-9B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=Self-Refine, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| ReActMemory=Base, Reflection=Reflexion, Backbone=Qwen2.5-27B2026.06 | 10 | |
| ReActMemory=Base, Reflection=Reflexion, Backbone=Qwen2.5-2B2026.06 | 10 | |
| ReActMemory=DC, Reflection=Reflexion, Backbone=Qwen2.5-27B2026.06 | 10 | |
| ReActMemory=DC, Reflection=Reflexion, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| ReActMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen2.5-2B2026.06 | 10 | |
| CoTMemory=Base, Reflection=Reflexion, Backbone=Qwen2.5-27B2026.06 | 10 | |
| CoTMemory=Base, Reflection=Reflexion, Backbone=Qwen2.5-9B2026.06 | 10 | |
| CoTMemory=Base, Reflection=Reflexion, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| CoTMemory=SimpleMem, Reflection=Reflexion, Backbone=Qwen2.5-9B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=Reflexion, Backbone=Qwen2.5-9B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen2.5-9B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen2.5-0.8B2026.06 | 10 | |
| NoneMemory=Base, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 0 | |
| NoneMemory=Base, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 0 | |
| NoneMemory=Base, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 0 | |
| ReActMemory=Base, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 0 | |
| ReActMemory=Base, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 0 | |
| ReActMemory=Base, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 0 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 0 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 0 | |
| ReActMemory=DC, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 0 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 0 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 0 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 0 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 0 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 0 | |
| CoTMemory=Base, Reflection=None, Backbone=Qwen2.5-27B2026.06 | 0 | |
| CoTMemory=Base, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 0 | |
| CoTMemory=Base, Reflection=None, Backbone=Qwen2.5-2B2026.06 | 0 | |
| CoTMemory=Base, Reflection=None, Backbone=Qwen2.5-0.8B2026.06 | 0 | |
| CoTMemory=Base, Reflection=None, Backbone=GPT-4o mini2026.06 | 0 | |
| CoTMemory=ChatDB, Reflection=None, Backbone=Qwen2.5-9B2026.06 | 0 |