Agent Task on AlfWorld
86.7Success RateResRL
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ResRLBackbone=Qwen2.5-7B-Instruct, Random seeds=32026.05 | 86.7 | — | 90.1 | 85.5 | 98 | 83 | 78.7 | 84.2 | — | — | — | — | |
| DeepSeek-R1Model Category=Thinking Model2026.02 | 83.6 | 64.1 | — | — | — | — | — | — | — | — | — | — | |
| SkillsInjector2026.05 | 82.7 | — | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.2Model Category=Non-thinking Model2026.02 | 80.7 | 59 | — | — | — | — | — | — | — | — | — | — | |
| Gemini 2.5 ProModel Category=Thinking Model2026.02 | 80.7 | 62.9 | — | — | — | — | — | — | — | — | — | — | |
| PPO (with critic)Backbone=Qwen2.5-7B-Instruct, Random seeds=32026.05 | 80.4 | — | 92.3 | 64 | 92.5 | 89.5 | 80.3 | 68.8 | — | — | — | — | |
| Gemini 2.5 FlashModel Category=Non-thinking Model2026.02 | 78.6 | 58.1 | — | — | — | — | — | — | — | — | — | — | |
| EMPGBackbone=Qwen2.5-7B-Instruct, Random seeds=32026.05 | 78.5 | — | 92.9 | 75.2 | 74.8 | 86.3 | 73.7 | 65.3 | — | — | — | — | |
| Graph of Skills2026.05 | 75.4 | — | — | — | — | — | — | — | — | — | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Random seeds=32026.05 | 74.8 | — | 88.8 | 43.7 | 88.1 | 70.3 | 77.7 | 56.8 | — | — | — | — | |
| SkillRouter2026.05 | 74.4 | — | — | — | — | — | — | — | — | — | — | — | |
| LLM-as-selector2026.05 | 73.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Dense Cosine2026.05 | 72.9 | — | — | — | — | — | — | — | — | — | — | — | |
| BM252026.05 | 71.2 | — | — | — | — | — | — | — | — | — | — | — | |
| Random-skill2026.05 | 69 | — | — | — | — | — | — | — | — | — | — | — | |
| No-skill2026.05 | 67.1 | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-30B-A3B-ThinkingModel Category=Thinking Model2026.02 | 66.4 | 50.7 | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-30B-A3B-InstructModel Category=Non-thinking Model2026.02 | 62.1 | 48.1 | — | — | — | — | — | — | — | — | — | — | |
| Gemini-2.5-ProBackbone=Gemini-2.5-Pro, Random seeds=32026.05 | 60.3 | — | 92.8 | 63.3 | 62.1 | 69 | 26.6 | 58.7 | — | — | — | — | |
| Llama-3.3-70B-InstructModel Category=Non-thinking Model2026.02 | 59.3 | 46.7 | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-4B-ThinkingModel Category=Thinking Model2026.02 | 57.1 | 40.9 | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-4B-InstructModel Category=Non-thinking Model2026.02 | 52.1 | 38.3 | — | — | — | — | — | — | — | — | — | — | |
| gpt-oss-120bModel Category=Thinking Model2026.02 | 50.7 | 29 | — | — | — | — | — | — | — | — | — | — | |
| GPT-4oBackbone=GPT-4o, Random seeds=32026.05 | 48 | — | 75.3 | 60.8 | 31.2 | 56.7 | 21.6 | 49.8 | — | — | — | — | |
| GLM-4-32B-0414Model Category=Non-thinking Model2026.02 | 37.9 | 29 | — | — | — | — | — | — | — | — | — | — | |
| COPESmall Model=Qwen3-8B, Large Model=GPT-4.12025.06 | 36.9 | — | — | — | — | — | — | — | 160 | — | — | — | |
| LargeModel=GPT-4.12025.06 | 35 | — | — | — | — | — | — | — | 225 | — | — | — | |
| Full-library2026.05 | 31.5 | — | — | — | — | — | — | — | — | — | — | — | |
| Prompting ReActBackbone=Qwen2.5-7B-Instruct, Random seeds=32026.05 | 31.2 | — | 48.5 | 35.4 | 34.3 | 13.2 | 18.2 | 17.6 | — | — | — | — | |
| GLM-4-9B-ChatModel Category=Non-thinking Model2026.02 | 25.7 | 21.8 | — | — | — | — | — | — | — | — | — | — | |
| Phi-4Model Category=Non-thinking Model2026.02 | 25 | 18.6 | — | — | — | — | — | — | — | — | — | — | |
| Ministral-3-14B-InstructModel Category=Non-thinking Model2026.02 | 22.1 | 15.8 | — | — | — | — | — | — | — | — | — | — | |
| Llama-3.1-8B-InstructModel Category=Non-thinking Model2026.02 | 16.4 | 12.8 | — | — | — | — | — | — | — | — | — | — | |
| GTRTeacher Model=GPT-4o, Training Time=164h, Cost Estimation=$145.76 / 30.94M tokens2025.12 | 16 | — | — | — | — | — | — | — | — | — | — | — | |
| Phi-4-reasoningModel Category=Thinking Model2026.02 | 15.7 | 9.8 | — | — | — | — | — | — | — | — | — | — | |
| GTR-TurboGuidance Mode=KL, Training Time=78h, Cost Estimation=$100.622025.12 | 15 | — | — | — | — | — | — | — | — | — | — | — | |
| Prompting BackboneBackbone=Qwen2.5-7B-Instruct, Random seeds=32026.05 | 14.8 | — | 33.4 | 21.6 | 19.3 | 6.9 | 2.8 | 3.2 | — | — | — | — | |
| GTR-TurboGuidance Mode=SFT, Training Time=118h, Cost Estimation=$152.222025.12 | 12 | — | — | — | — | — | — | — | — | — | — | — | |
| Mistral-7B-InstructModel Category=Non-thinking Model2026.02 | 8.6 | 7.6 | — | — | — | — | — | — | — | — | — | — | |
| RL4VLMTraining Time=70h, Cost Estimation=$02025.12 | 8 | — | — | — | — | — | — | — | — | — | — | — | |
| DAPOBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 82 | 89.8 | 91.2 | |
| Gemini-2.5-ProBackbone=Gemini-2.5-Pro2026.07 | — | — | — | — | — | — | — | — | — | 54.3 | 61.2 | 64.8 | |
| GiGPOBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 85.9 | 87.5 | 89.8 | |
| GPT-4oBackbone=GPT-4o2026.07 | — | — | — | — | — | — | — | — | — | 42.5 | 48.3 | 50.8 | |
| GRPOBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 76.4 | 80.3 | 82.6 | |
| GSPOBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 85.9 | 91.4 | 91.4 | |
| Mem0+GRPOBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 82.8 | 84.7 | 85.2 | |
| MetaRLBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 82 | 85.9 | 86.7 | |
| PivoARLBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 89.8 | 93.4 | 94.5 | |
| ReActBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 17.1 | 21.8 | 27.2 | |
| ReflexionBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 17.9 | 28.1 | 29.7 | |
| SimpleMem+GRPOBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 87.5 | 89 | 90.6 | |
| SKillRLBackbone=Qwen3-4B2026.07 | — | — | — | — | — | — | — | — | — | 83.6 | 91.4 | 92.5 |