Embodied AI Instruction Following on ALFRED (test)
70Success Rate (SR)CoT
Evaluation results
| Method | Links | |
|---|---|---|
| CoTMemory=SimpleMem, Reflection=Reflexion, Backbone=Qwen3.5-27B2026.06 | 70 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 60 | |
| CoTMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 60 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 50 | |
| ReActMemory=DC, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 50 | |
| CoTMemory=SimpleMem, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 50 | |
| ReActMemory=Base, Reflection=Reflexion, Backbone=Qwen3.5-27B2026.06 | 50 | |
| ReActMemory=Base, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 50 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 50 | |
| ReActMemory=Base, Reflection=Self-Refine, Backbone=GPT-5 mini2026.06 | 50 | |
| ReActMemory=Base, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 40 | |
| CoTMemory=DC, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 40 | |
| CoTMemory=MemoryBank, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 40 | |
| MADMemory=Base, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 40 | |
| ReActMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen3.5-27B2026.06 | 40 | |
| CoTMemory=Base, Reflection=Reflexion, Backbone=Qwen3.5-27B2026.06 | 40 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 40 | |
| CoTMemory=Base, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 40 | |
| CoTMemory=DC, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 40 | |
| CoTMemory=Base, Reflection=Self-Refine, Backbone=Qwen3.5-9B2026.06 | 40 | |
| MADMemory=MemoryBank, Reflection=Self-Refine, Backbone=Qwen3.5-9B2026.06 | 40 | |
| CoTMemory=SimpleMem, Reflection=Reflexion, Backbone=Qwen3.5-9B2026.06 | 40 | |
| ReActMemory=Base, Reflection=None, Backbone=GPT-5 mini2026.06 | 40 | |
| ReActMemory=ChatDB, Reflection=None, Backbone=GPT-5 mini2026.06 | 40 | |
| ReActMemory=DC, Reflection=None, Backbone=GPT-5 mini2026.06 | 40 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 30 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 30 | |
| CoTMemory=Base, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 30 | |
| Plan&SolveMemory=MemoryBank, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 30 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| CoTMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| CoTMemory=SimpleMem, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| CoTMemory=MemoryBank, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| Plan&SolveMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| MADMemory=Base, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| MADMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| MADMemory=DC, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 30 | |
| ReActMemory=Base, Reflection=Self-Refine, Backbone=Qwen3.5-9B2026.06 | 30 | |
| CoTMemory=SimpleMem, Reflection=Self-Refine, Backbone=Qwen3.5-9B2026.06 | 30 | |
| ReActMemory=Base, Reflection=Reflexion, Backbone=Qwen3.5-9B2026.06 | 30 | |
| ReActMemory=DC, Reflection=Reflexion, Backbone=Qwen3.5-9B2026.06 | 30 | |
| ReActMemory=Base, Reflection=Reflexion, Backbone=GPT-5 mini2026.06 | 30 | |
| Plan&SolveMemory=SimpleMem, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 20 | |
| CoTMemory=Base, Reflection=Self-Refine, Backbone=Qwen3.5-27B2026.06 | 20 | |
| ReActMemory=DC, Reflection=Reflexion, Backbone=Qwen3.5-27B2026.06 | 20 | |
| MADMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen3.5-27B2026.06 | 20 | |
| ReActMemory=DC, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 20 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 20 | |
| CoTMemory=Base, Reflection=Reflexion, Backbone=Qwen3.5-9B2026.06 | 20 | |
| ReActMemory=SimpleMem, Reflection=None, Backbone=GPT-5 mini2026.06 | 20 | |
| CoTMemory=Base, Reflection=None, Backbone=GPT-5 mini2026.06 | 20 | |
| Plan&SolveMemory=Base, Reflection=None, Backbone=GPT-5 mini2026.06 | 20 | |
| Plan&SolveMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 10 | |
| Plan&SolveMemory=DC, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 10 | |
| MADMemory=ChatDB, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 10 | |
| MADMemory=DC, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 10 | |
| ReActMemory=Base, Reflection=Self-Refine, Backbone=Qwen3.5-27B2026.06 | 10 | |
| CoTMemory=SimpleMem, Reflection=Self-Refine, Backbone=Qwen3.5-27B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=Self-Refine, Backbone=Qwen3.5-27B2026.06 | 10 | |
| MADMemory=MemoryBank, Reflection=Self-Refine, Backbone=Qwen3.5-27B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=Self-Refine, Backbone=Qwen3.5-9B2026.06 | 10 | |
| ReActMemory=MemoryBank, Reflection=Reflexion, Backbone=Qwen3.5-9B2026.06 | 10 | |
| Plan&SolveMemory=Base, Reflection=Reflexion, Backbone=Qwen3.5-9B2026.06 | 10 | |
| ReActMemory=MemoryBank, Reflection=None, Backbone=GPT-5 mini2026.06 | 10 | |
| ReActMemory=OpenClaw, Reflection=None, Backbone=GPT-5 mini2026.06 | 10 | |
| NoneMemory=Base, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 0 | |
| Plan&SolveMemory=Base, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 0 | |
| MADMemory=SimpleMem, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 0 | |
| MADMemory=MemoryBank, Reflection=None, Backbone=Qwen3.5-27B2026.06 | 0 | |
| Plan&SolveMemory=Base, Reflection=Reflexion, Backbone=Qwen3.5-27B2026.06 | 0 | |
| NoneMemory=Base, Reflection=None, Backbone=Qwen3.5-9B2026.06 | 0 | |
| NoneMemory=Base, Reflection=None, Backbone=GPT-5 mini2026.06 | 0 | |
| MADMemory=Base, Reflection=None, Backbone=GPT-5 mini2026.06 | 0 |