Multi-turn embodied reasoning on BabyAI
73Success RateReMem
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReMemLLM Backbone=Claude 3.7 Sonnet2025.11 | 73 | 83 | — | |
| ExpRecentLLM Backbone=Claude 3.7 Sonnet2025.11 | 63 | 73 | — | |
| ExpRAGLLM Backbone=Claude 3.7 Sonnet2025.11 | 62 | 72 | — | |
| BaselineLLM Backbone=Gemini 2.5 Flash2025.11 | 61 | 71 | — | |
| ReActLLM Backbone=Claude 3.7 Sonnet2025.11 | 57 | 72 | — | |
| ExpRAGLLM Backbone=Gemini 2.5 Flash2025.11 | 56 | 65 | — | |
| Qwen2.5-72BActor=Qwen2.5-72B, Thinker=No2026.05 | 55.35 | — | 70.07 | |
| Mem0LLM Backbone=Gemini 2.5 Flash2025.11 | 54 | 66 | — | |
| AmemLLM Backbone=Gemini 2.5 Flash2025.11 | 53 | 64 | — | |
| DC-CuLLM Backbone=Gemini 2.5 Flash2025.11 | 53 | 64 | — | |
| DC-RSLLM Backbone=Gemini 2.5 Flash2025.11 | 53 | 66 | — | |
| ExpRecentLLM Backbone=Gemini 2.5 Flash2025.11 | 53 | 64 | — | |
| ReMemLLM Backbone=Gemini 2.5 Flash2025.11 | 53 | 61 | — | |
| AWMLLM Backbone=Claude 3.7 Sonnet2025.11 | 53 | 68 | — | |
| HistoryLLM Backbone=Gemini 2.5 Flash2025.11 | 52 | 64 | — | |
| SelfRAGLLM Backbone=Gemini 2.5 Flash2025.11 | 52 | 65 | — | |
| AWMLLM Backbone=Gemini 2.5 Flash2025.11 | 52 | 64 | — | |
| DC-RSLLM Backbone=Claude 3.7 Sonnet2025.11 | 52 | 68 | — | |
| BaselineLLM Backbone=Claude 3.7 Sonnet2025.11 | 51 | 66 | — | |
| DC-CuLLM Backbone=Claude 3.7 Sonnet2025.11 | 50 | 67 | — | |
| Qwen2.5-7BActor=Qwen2.5-7B, Thinker=Exp-Thinker2026.05 | 50 | — | 63.12 | |
| Qwen2.5-ActorActor=Qwen2.5-Actor, Thinker=Exp-Thinker2026.05 | 48.21 | — | 60.25 | |
| GPT-4oActor=GPT-4o, Thinker=No2026.05 | 48.2 | — | 64.1 | |
| ReActLLM Backbone=Gemini 2.5 Flash2025.11 | 48 | 63 | — | |
| HistoryLLM Backbone=Claude 3.7 Sonnet2025.11 | 48 | 66 | — | |
| Mem0LLM Backbone=Claude 3.7 Sonnet2025.11 | 48 | 66 | — | |
| AgentGymActor=AgentGym, Thinker=No2026.05 | 47.3 | — | 61.4 | |
| AmemLLM Backbone=Claude 3.7 Sonnet2025.11 | 46 | 64 | — | |
| SelfRAGLLM Backbone=Claude 3.7 Sonnet2025.11 | 46 | 64 | — | |
| LLaMA3-8BActor=LLaMA3-8B, Thinker=Exp-Thinker2026.05 | 41.96 | — | 58.15 | |
| Qwen2.5-ActorActor=Qwen2.5-Actor, Thinker=No2026.05 | 40.17 | — | 50.62 | |
| AgentRefineActor=AgentRefine, Thinker=No2026.05 | 37.5 | — | 50.4 | |
| Qwen2.5-7BActor=Qwen2.5-7B, Thinker=No2026.05 | 37.5 | — | 50.16 | |
| LLaMA3-ActorActor=LLaMA3-Actor, Thinker=Exp-Thinker2026.05 | 35.71 | — | 46.41 | |
| LLaMA3-8BActor=LLaMA3-8B, Thinker=No2026.05 | 26.78 | — | 42.44 | |
| Agent-FLANActor=Agent-FLAN, Thinker=No2026.05 | 25 | — | 35.3 | |
| LLaMA3-ActorActor=LLaMA3-Actor, Thinker=No2026.05 | 23.21 | — | 35.55 |