Embodied Task on ALFWorld
97.5Overall Success RateSkill1
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Skill1Base Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=Yes2026.05 | 97.5 | 100 | 98.6 | 97.3 | 99.2 | 96.1 | 96 | — | — | |
| GRPO w/ SG & GRBackbone=Qwen3-4B, Setting=RL Training2026.04 | 96.9 | 100 | 100 | 100 | 100 | 87 | 93.8 | — | — | |
| GRPO w/ SG & GRBackbone=Qwen2.5-7B-Instruct, Setting=RL Training2026.04 | 95.3 | 100 | 100 | 100 | 100 | 90.5 | 84 | — | — | |
| RetroAgentBase Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=Yes2026.05 | 94.9 | 97.9 | 90.9 | 99.2 | 92.9 | 85.3 | 91 | — | — | |
| R3LBackbone=Qwen2.5-7B-Instruct, Category=RL Methods2026.03 | 94.8 | — | — | — | — | — | — | — | — | |
| ENVRL-GiGPOBackbone Model=Qwen2.5-7B-Instruct2026.06 | 94.5 | 98.3 | 92.2 | 98.9 | 93.8 | 93.3 | 94.6 | — | — | |
| SLEA-RLBackbone=Qwen2.5-7B-Instruct, Category=RL Methods2026.03 | 93.5 | 98.5 | 85.1 | 99 | 93.8 | 96.4 | 90.4 | — | — | |
| SLEA-RLBase Model=Qwen2.5-7B-Instruct, Type=RL Methods2026.03 | 93.5 | 98.5 | 85.1 | 99 | 93.8 | 96.4 | 90.4 | — | — | |
| SIRICategory=Memory-Augmented RL-based Methods, Backbone=Qwen2.5-7B-Instruct2026.06 | 93 | 97.5 | 87.5 | 100 | 85.7 | 85 | 89.5 | — | — | |
| GRPO w/ SGBackbone=Qwen2.5-7B-Instruct, Setting=RL Training2026.04 | 92.2 | 100 | 91.7 | 100 | 89.5 | 100 | 70.8 | — | — | |
| HiMACType=RL, Backbone=Qwen2.5-7B-Instruct2026.03 | 92.1 | 99.1 | 88.1 | 98.4 | 91.9 | 92.7 | 89.5 | — | — | |
| ENVRL-GiGPOBackbone Model=Qwen2.5-1.5B-Instruct2026.06 | 91.8 | 95 | 70.8 | 94.9 | 97.7 | 94.1 | 94.7 | — | — | |
| GRPO w/ SGBackbone=Qwen3-4B, Setting=RL Training2026.04 | 91.4 | 97.1 | 80 | 96.2 | 100 | 87.5 | 80 | — | — | |
| GiGPOw/stdType=RL, Backbone=Qwen2.5-7B-Instruct2026.03 | 90.8 | 97.7 | 82.7 | 97.8 | 83.7 | 89.3 | 79.2 | — | — | |
| GiGPOBackbone=Qwen2.5-7B-Instruct, Category=RL Methods2026.03 | 90.8 | 97.7 | 82.7 | 98.8 | 83.7 | 89.3 | 79.2 | — | — | |
| GiGPO*Base Model=Qwen2.5-7B-Instruct, Type=RL Methods2026.03 | 90.8 | 97.7 | 82.7 | 98.8 | 83.7 | 89.3 | 79.2 | — | — | |
| GiGPOBase Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=No2026.05 | 90.8 | 97.7 | 82.7 | 98.8 | 83.7 | 89.3 | 79.2 | — | — | |
| GiGPOCategory=RL-based Methods, Backbone=Qwen2.5-7B-Instruct2026.06 | 90.8 | 97.7 | 82.7 | 98.8 | 83.7 | 89.3 | 79.2 | — | — | |
| GiGPOBackbone Model=Qwen2.5-7B-Instruct2026.06 | 90.8 | 97.7 | 82.7 | 98.8 | 83.7 | 89.3 | 79.2 | — | — | |
| GiGPOw/ostdType=RL, Backbone=Qwen2.5-7B-Instruct2026.03 | 90.2 | 91.8 | 88.6 | 95.9 | 90.2 | 86.5 | 85.2 | — | — | |
| HiMACType=RL, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 89.9 | 98.6 | 79.9 | 98.2 | 90.5 | 80.1 | 84.3 | — | — | |
| SkillRLBackbone=Qwen2.5-7B-Instruct, Category=RL Methods2026.03 | 89.9 | 97.9 | 71.4 | 90 | 90 | 95.5 | 87.5 | — | — | |
| SkillRLBase Model=Qwen2.5-7B-Instruct, Type=RL Methods2026.03 | 89.9 | 97.9 | 71.4 | 90 | 90 | 95.5 | 87.5 | — | — | |
| SkillRLBase Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=Yes2026.05 | 89.9 | 97.9 | 71.4 | 90 | 90 | 95.5 | 87.5 | — | — | |
| SkillRLCategory=Memory-Augmented RL-based Methods, Backbone=Qwen2.5-7B-Instruct2026.06 | 89.9 | 97.9 | 71.4 | 90 | 90 | 95.5 | 87.5 | — | — | |
| SKILL0Backbone=Qwen2.5-(VL)-7B-Instruct2026.04 | 89.8 | 100 | 85.8 | 94.6 | 81.9 | 85.7 | 80.1 | — | — | |
| GRPO w/ SG & GRBackbone=Qwen3-1.7B, Setting=RL Training2026.04 | 89.8 | 100 | 100 | 78.1 | 100 | 78.8 | 100 | — | — | |
| SKILL0Backbone=Qwen2.5-(VL)-3B-Instruct2026.04 | 87.9 | 95.6 | 80.4 | 100 | 86.7 | 78.7 | 75.2 | — | — | |
| SLEA-RLBackbone=Qwen2.5-1.5B-Instruct2026.03 | 87.5 | 92.6 | 80 | 96.2 | 94.7 | 95 | 76.5 | — | — | |
| SLEA-RLBase Model=Qwen2.5-1.5B-Instruct2026.03 | 87.5 | 92.6 | 80 | 96.2 | 94.7 | 95 | 76.5 | — | — | |
| GiGPOw/stdType=RL, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 86.7 | 94.4 | 67.5 | 94.8 | 94.4 | 79.8 | 76.4 | — | — | |
| GiGPOBackbone=Qwen2.5-1.5B-Instruct2026.03 | 86.7 | 94.4 | 67.5 | 94.8 | 94.4 | 79.8 | 76.4 | — | — | |
| GiGPO*Base Model=Qwen2.5-1.5B-Instruct2026.03 | 86.7 | 94.4 | 67.5 | 94.8 | 94.4 | 79.8 | 76.4 | — | — | |
| GRPOBackbone=Qwen3-4B, Setting=RL Training2026.04 | 86.7 | 95.7 | 88.9 | 100 | 92.3 | 67.7 | 86.4 | — | — | |
| GiGPOBackbone Model=Qwen2.5-1.5B-Instruct2026.06 | 86.7 | 94.4 | 67.5 | 94.8 | 94.4 | 79.8 | 76.4 | — | — | |
| GiGPOw/ostdType=RL, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 86.1 | 96 | 76.5 | 91.8 | 91.3 | 71.7 | 79.5 | — | — | |
| Gemini-3-FlashCategory=Closed-source LLMs2026.06 | 85.2 | 96.4 | 85.7 | 57.1 | 72.2 | 96.2 | 95.3 | — | — | |
| SDARBackbone=Qwen2.5-3B-Instruct2026.05 | 84.4 | 97.1 | 62.5 | 100 | 61.9 | 75 | 84.2 | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Setting=RL Training2026.04 | 83.6 | 100 | 72.7 | 100 | 88.9 | 58.8 | 89.5 | — | — | |
| StepOPSDBackbone=Qwen2.5-3B-Instruct, lambda_mix=0.2, alpha_clip=0.052026.05 | 83.6 | 97.1 | 66.7 | 87 | 79.1 | 78.9 | 95 | — | — | |
| Claude Sonnet 4.5Type=PE, Backbone=Closed-Source Model2026.03 | 82.5 | — | — | — | — | — | — | — | — | |
| GRPO w/ SGBackbone=Qwen3-1.7B, Setting=RL Training2026.04 | 81.3 | 94.1 | 100 | 66.7 | 76 | 68 | 83.3 | — | — | |
| AgentOCRBackbone=Qwen2.5-(VL)-7B-Instruct2026.04 | 81.2 | 95.6 | 96.2 | 78.1 | 73.2 | 72.4 | 72 | — | — | |
| GRPO+OPSDBackbone=Qwen2.5-3B-Instruct2026.05 | 81.2 | 100 | 82.4 | 85.7 | 75 | 70 | 60 | — | — | |
| Skill-GRPO*Backbone=Qwen2.5-3B-Instruct2026.05 | 80.5 | 94.3 | 57.1 | 100 | 66.7 | 73.1 | 57.1 | — | — | |
| PPOType=RL, Backbone=Qwen2.5-7B-Instruct2026.03 | 80.4 | 92.3 | 64 | 92.5 | 89.5 | 80.3 | 68.8 | — | — | |
| PPOBase Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=No2026.05 | 80.4 | 92.3 | 64 | 92.5 | 89.5 | 80.3 | 68.8 | — | — | |
| PPOCategory=RL-based Methods, Backbone=Qwen2.5-7B-Instruct2026.06 | 80.4 | 92.3 | 64 | 92.5 | 89.5 | 80.3 | 68.8 | — | — | |
| ENVRL-GRPOBackbone Model=Qwen2.5-7B-Instruct2026.06 | 80.4 | 92.6 | 77.5 | 92.9 | 79.7 | 77.4 | 71.1 | — | — | |
| SkillOSExecutor=Gemini-2.5-pro, Curator=Qwen3-8B2026.05 | 80.2 | 95.2 | 71.8 | 74.1 | 72.9 | 77.3 | 77.8 | — | 14.8 | |
| GRPOBackbone=Qwen2.5-(VL)-3B-Instruct2026.04 | 79.9 | 92.6 | 85.7 | 70.6 | 86.6 | 79.3 | 65 | — | — | |
| RLSDBackbone=Qwen2.5-3B-Instruct2026.05 | 79.7 | 87.9 | 75 | 90.9 | 75 | 73.1 | 68.4 | — | — | |
| GLM-5.1Category=Closed-source LLMs2026.06 | 79.7 | 91.3 | 81.8 | 85.2 | 93.8 | 66.7 | 66.7 | — | — | |
| SkillOS-geminiExecutor=Gemini-2.5-pro, Curator=Gemini-2.5-Pro2026.05 | 79.3 | 94.3 | 69.2 | 77.8 | 75 | 80 | 66.7 | — | 14.9 | |
| AgentOCRBackbone=Qwen2.5-(VL)-3B-Instruct2026.04 | 78.2 | 91.9 | 81.8 | 76 | 73.3 | 76.1 | 70 | — | — | |
| GRPOType=RL, Backbone=Qwen2.5-7B-Instruct2026.03 | 77.6 | 90.8 | 66.1 | 89.3 | 74.7 | 72.5 | 64.7 | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Category=RL Methods2026.03 | 77.6 | 90.8 | 66.1 | 89.3 | 74.7 | 72.5 | 64.7 | — | — | |
| GRPO*Base Model=Qwen2.5-7B-Instruct, Type=RL Methods2026.03 | 77.6 | 90.8 | 66.1 | 89.3 | 74.7 | 72.5 | 64.7 | — | — | |
| GRPOBackbone=Qwen2.5-(VL)-7B-Instruct2026.04 | 77.6 | 90.8 | 66.1 | 89.3 | 74.7 | 72.5 | 64.7 | — | — | |
| GRPOBase Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=No2026.05 | 77.6 | 90.8 | 66.1 | 89.3 | 74.7 | 72.5 | 64.7 | — | — | |
| GRPOCategory=RL-based Methods, Backbone=Qwen2.5-7B-Instruct2026.06 | 77.6 | 90.8 | 66.1 | 89.3 | 74.7 | 72.5 | 64.7 | — | — | |
| GRPOBackbone Model=Qwen2.5-7B-Instruct2026.06 | 77.6 | 90.8 | 66.1 | 89.3 | 74.7 | 72.5 | 64.7 | — | — | |
| ENVRL-GRPOBackbone Model=Qwen2.5-1.5B-Instruct2026.06 | 77.4 | 85.9 | 53.9 | 85 | 82 | 76.8 | 71.9 | — | — | |
| RLOOType=RL, Backbone=Qwen2.5-7B-Instruct2026.03 | 75.5 | 87.6 | 78.2 | 87.3 | 81.3 | 71.9 | 48.9 | — | — | |
| RLOOBackbone=Qwen2.5-7B-Instruct, Category=RL Methods2026.03 | 75.5 | 87.6 | 78.2 | 87.3 | 81.3 | 71.9 | 48.9 | — | — | |
| RLOO*Base Model=Qwen2.5-7B-Instruct, Type=RL Methods2026.03 | 75.5 | 87.6 | 78.2 | 87.3 | 81.3 | 71.9 | 48.9 | — | — | |
| RLOOBackbone=Qwen2.5-(VL)-7B-Instruct2026.04 | 75.5 | 87.6 | 78.2 | 87.3 | 81.3 | 71.9 | 48.9 | — | — | |
| RLOOBase Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=No2026.05 | 75.5 | 87.6 | 78.2 | 87.3 | 81.3 | 71.9 | 48.9 | — | — | |
| RLOOCategory=RL-based Methods, Backbone=Qwen2.5-7B-Instruct2026.06 | 75.5 | 87.6 | 78.2 | 87.3 | 81.3 | 71.9 | 48.9 | — | — | |
| MemPExecutor=Gemini-2.5-pro, Curator=Qwen3-8B2026.05 | 74.3 | 95.2 | 74.4 | 61.7 | 56.3 | 76 | 68.1 | — | 15.2 | |
| Skill-SDBackbone=Qwen2.5-3B-Instruct2026.05 | 73.4 | 88.2 | 50 | 96.2 | 52.4 | 65 | 57.9 | — | — | |
| StepOPSDBackbone=Qwen2.5-3B-Instruct, lambda_mix=0.052026.05 | 73.4 | 82.4 | 66.7 | 82.6 | 52.2 | 73.7 | 75 | — | — | |
| GRPOType=RL, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 72.8 | 85.3 | 53.7 | 84.5 | 78.2 | 59.7 | 53.5 | — | — | |
| GRPOBackbone=Qwen2.5-1.5B-Instruct2026.03 | 72.8 | 85.3 | 53.7 | 84.5 | 78.2 | 59.7 | 53.5 | — | — | |
| GRPO*Base Model=Qwen2.5-1.5B-Instruct2026.03 | 72.8 | 85.3 | 53.7 | 84.5 | 78.2 | 59.7 | 53.5 | — | — | |
| GRPOBackbone Model=Qwen2.5-1.5B-Instruct2026.06 | 72.8 | 85.3 | 53.7 | 84.5 | 78.2 | 59.7 | 53.5 | — | — | |
| GRPOBackbone=Qwen3-1.7B, Setting=RL Training2026.04 | 72.7 | 93.1 | 100 | 83.3 | 52.6 | 54.5 | 60.7 | — | — | |
| ReasoningBankExecutor=Gemini-2.5-pro, Curator=Qwen3-8B2026.05 | 71.4 | 91.4 | 61.5 | 63 | 39.6 | 70.7 | 76.4 | — | 16 | |
| SkillOS-baseExecutor=Gemini-2.5-pro, Curator=Qwen3-8B2026.05 | 70.7 | 91.4 | 69.2 | 56.8 | 54.2 | 72 | 66.7 | — | 16.3 | |
| GRPOBackbone=Qwen2.5-3B-Instruct2026.05 | 70 | 91.2 | 62.5 | 96.2 | 61.9 | 65 | 47.4 | — | — | |
| RLOOType=RL, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 69.7 | 88.3 | 52.8 | 71 | 62.8 | 66.4 | 56.9 | — | — | |
| StepOPSDBackbone=Qwen2.5-3B-Instruct, lambda_mix=0.22026.05 | 69.5 | 45 | 55.6 | 87 | 65.2 | 57.9 | 85.3 | — | — | |
| SkillOSExecutor=Qwen3-32B, Curator=Qwen3-8B2026.05 | 68.6 | 91.4 | 76.9 | 59.3 | 56.3 | 57.3 | 62.5 | — | 17.3 | |
| No MemoryExecutor=Gemini-2.5-pro, Curator=None2026.05 | 66.4 | 90.5 | 66.7 | 48.1 | 39.6 | 68.4 | 68.1 | — | 17.7 | |
| GPT-5Type=PE, Backbone=Closed-Source Model2026.03 | 65.4 | — | — | — | — | — | — | — | — | |
| SkillOS-geminiExecutor=Qwen3-32B, Curator=Gemini-2.5-Pro2026.05 | 63.6 | 97.1 | 76.9 | 55.6 | 43.8 | 40 | 54.2 | — | 18.1 | |
| SimpleMem+GRPOBackbone=Qwen2.5-7B-Instruct, Category=RL Methods2026.03 | 62.5 | 89.5 | 36.3 | 60 | 50 | 64.9 | 26.3 | — | — | |
| SimpleMem+GRPOBase Model=Qwen2.5-7B-Instruct, Type=RL Methods2026.03 | 62.5 | 89.5 | 36.3 | 60 | 50 | 64.9 | 26.3 | — | — | |
| SimpleMemBackbone=Qwen2.5-(VL)-7B-Instruct, Training Strategy=GRPO2026.04 | 62.5 | 89.5 | 36.3 | 60 | 50 | 64.9 | 26.3 | — | — | |
| SimpleMem w/ GRPOBase Model=Qwen2.5-7B-Instruct, Training Strategy=RL-Trained, Skill Library=Yes2026.05 | 62.5 | 89.5 | 36.3 | 60 | 50 | 64.9 | 26.3 | — | — | |
| SimpleMem+GRPOCategory=Memory-Augmented RL-based Methods, Backbone=Qwen2.5-7B-Instruct2026.06 | 62.5 | 89.5 | 36.3 | 60 | 50 | 64.9 | 26.3 | — | — | |
| ReasoningBankExecutor=Qwen3-32B, Curator=Qwen3-8B2026.05 | 61.4 | 86.7 | 71.8 | 50.6 | 45.8 | 52 | 51.4 | — | 18.7 | |
| SkillOSExecutor=Qwen3-8B, Curator=Qwen3-8B2026.05 | 61.2 | 85.7 | 56.4 | 54.3 | 43.8 | 46.7 | 62.5 | — | 18.9 | |
| Gemini-2.5-ProType=PE, Backbone=Closed-Source Model2026.03 | 60.3 | 92.8 | 63.3 | 62.1 | 69 | 26.6 | 58.7 | — | — | |
| Gemini-2.5-ProType=Closed-Source LLMs2026.03 | 60.3 | 92.8 | 63.3 | 62.1 | 69 | 26.6 | 58.7 | — | — | |
| Gemini-2.5-ProBackbone=Closed-source LLMs2026.04 | 60.3 | 92.8 | 63.3 | 62.1 | 69 | 26.6 | 58.7 | — | — | |
| Gemini-2.5-ProBackbone Model=Proprietary Model2026.06 | 60.3 | 92.8 | 63.3 | 62.1 | 69 | 26.6 | 58.7 | — | — | |
| Skill-GRPOBackbone=Qwen2.5-3B-Instruct2026.05 | 60.2 | 98.9 | 71.4 | 58.8 | 70.6 | 40.7 | 29.2 | — | — | |
| SkillOS-baseExecutor=Qwen3-32B, Curator=Qwen3-8B2026.05 | 59.8 | 82.9 | 69.2 | 48.1 | 50 | 48 | 52.8 | — | 19.2 | |
| ReAct w/ SGBackbone=Qwen3-4B, Setting=Prompting2026.04 | 58.6 | 87.8 | 50 | 29.2 | 58.3 | 27.3 | 80 | — | — |