Online Shopping on Webshop
91.5ScoreSKILLGRAPH
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| SKILLGRAPHCategory=Ours2026.05 | 91.5 | — | — | — | — | 84.4 | — | — | — | — | |
| RSPO+GiGPOBase Model=Qwen2.5-7B-Instruct2026.07 | 89 | — | — | — | — | 77.3 | — | — | — | — | |
| GiGPOType=RL Training, Backbone=Qwen2.5-(VL)-7B-Instruct2026.03 | 88.4 | — | — | — | — | 72.7 | — | — | — | — | |
| EMPO2Base Model=Qwen2.5-7B-Instruct, Learning Paradigm=Online RL, Test-time Memory=false2026.02 | 88.3 | — | — | — | — | 76.9 | — | — | — | — | |
| RSPO+GiGPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 88.2 | — | — | — | — | 75.3 | — | — | — | — | |
| Role-AgentType=RL Training, Backbone=Qwen2.5-7B-Instruct2026.06 | 88 | — | — | — | — | 77.1 | — | — | — | — | |
| Role-AgentType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.06 | 87.7 | — | — | — | — | 71.9 | — | — | — | — | |
| GiGPOBase Model=Qwen2.5-7B-Instruct2026.07 | 86.8 | — | — | — | — | 75.3 | — | — | — | — | |
| GiGPO w/o stdBase Model=Qwen2.5-7B-Instruct, Learning Paradigm=Online RL, Normalization Factor (Fnorm)=12026.02 | 86.2 | — | — | — | — | 75.2 | — | — | — | — | |
| RSPO+GRPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 85.8 | — | — | — | — | 70.6 | — | — | — | — | |
| RSPO+PPOBase Model=Qwen2.5-7B-Instruct2026.07 | 85.3 | — | — | — | — | 74 | — | — | — | — | |
| SkillRLCategory=Memory-Augmented RL-based Methods2026.05 | 85.2 | — | — | — | — | 72.7 | — | — | — | — | |
| RSPO+GRPOBase Model=Qwen2.5-7B-Instruct2026.07 | 84.8 | — | — | — | — | 75 | — | — | — | — | |
| PPOBase Model=Qwen2.5-7B-Instruct2026.07 | 84.7 | — | — | — | — | 70.6 | — | — | — | — | |
| GiGPO w/ stdBase Model=Qwen2.5-7B-Instruct, Learning Paradigm=Online RL, Normalization Factor (Fnorm)=std2026.02 | 84.4 | — | — | — | — | 72.8 | — | — | — | — | |
| RewardFlowType=RL Training, Backbone=Qwen2.5-(VL)-7B-Instruct2026.03 | 84.4 | — | — | — | — | 73.4 | — | — | — | — | |
| GiGPOType=RL Training, Backbone=Qwen2.5-7B-Instruct2026.06 | 84.4 | — | — | — | — | 72.8 | — | — | — | — | |
| RSPO+PPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 84.3 | — | — | — | — | 69.3 | — | — | — | — | |
| RLOOType=RL Training, Backbone=Qwen2.5-(VL)-7B-Instruct2026.03 | 84.2 | — | — | — | — | 72.7 | — | — | — | — | |
| AAWMBackbone=Qwen2.5-7B-Instruct, Training Stage=Imitation Learning then Reinforcement Learning initialized by World Modeling2026.06 | 84.2 | — | — | — | — | 76.6 | — | — | — | — | |
| SPEAR+GiGPOBase Model=Qwen2.5-7B-Instruct2026.07 | 83.5 | — | — | — | — | 67.7 | — | — | — | — | |
| GRPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 83.3 | — | — | — | — | 65.1 | — | — | — | — | |
| GiGPOType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.06 | 83.1 | — | — | — | — | 65 | — | — | — | — | |
| GiGPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 82.5 | — | — | — | — | 67.4 | — | — | — | — | |
| IWMBackbone=Qwen2.5-7B-Instruct, Training Stage=Imitation Learning then Reinforcement Learning initialized by World Modeling2026.06 | 82.2 | — | — | — | — | 73.2 | — | — | — | — | |
| RewardFlowType=RL Training, Backbone=Qwen2.5-(VL)-3B-Instruct2026.03 | 81.8 | — | — | — | — | 60.9 | — | — | — | — | |
| AAWMBackbone=Qwen2.5-1.5B-Instruct, Training Stage=Imitation Learning then Reinforcement Learning initialized by World Modeling2026.06 | 81.5 | — | — | — | — | 73.4 | — | — | — | — | |
| PPOType=RL Training, Backbone=Qwen2.5-7B-Instruct2026.06 | 81.4 | — | — | — | — | 68.7 | — | — | — | — | |
| RLOOType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 80.9 | — | — | — | — | 54.7 | — | — | — | — | |
| BaseBackbone=Qwen2.5-7B-Instruct, Training Stage=Imitation Learning then Reinforcement Learning initialized by World Modeling2026.06 | 80.8 | — | — | — | — | 70.6 | — | — | — | — | |
| RLOOCategory=RL-based Methods2026.05 | 80.3 | — | — | — | — | 65.7 | — | — | — | — | |
| RLOOType=RL Training, Backbone=Qwen2.5-7B-Instruct2026.06 | 80.3 | — | — | — | — | 65.7 | — | — | — | — | |
| GiGPOType=RL Training, Backbone=Qwen2.5-(VL)-3B-Instruct2026.03 | 80 | — | — | — | — | 59.4 | — | — | — | — | |
| GRPOBase Model=Qwen2.5-7B-Instruct, Learning Paradigm=Online RL2026.02 | 79.3 | — | — | — | — | 66.1 | — | — | — | — | |
| GRPOCategory=RL-based Methods2026.05 | 79.3 | — | — | — | — | 66.1 | — | — | — | — | |
| GRPOType=RL Training, Backbone=Qwen2.5-7B-Instruct2026.06 | 79.3 | — | — | — | — | 66.1 | — | — | — | — | |
| RewardFlowType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 78.3 | — | — | — | — | 60.9 | — | — | — | — | |
| GRPOBase Model=Qwen2.5-7B-Instruct2026.07 | 78.3 | — | — | — | — | 68.5 | — | — | — | — | |
| IWMBackbone=Qwen2.5-1.5B-Instruct, Training Stage=Imitation Learning then Reinforcement Learning initialized by World Modeling2026.06 | 77 | — | — | — | — | 67.2 | — | — | — | — | |
| SPEAR+GRPOBase Model=Qwen2.5-7B-Instruct2026.07 | 76.7 | — | — | — | — | 61.5 | — | — | — | — | |
| GRPOType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.06 | 75.8 | — | — | — | — | 56.8 | — | — | — | — | |
| GiGPOType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 75.4 | — | — | — | — | 55.5 | — | — | — | — | |
| RLOOType=RL Training, Backbone=Qwen2.5-(VL)-3B-Instruct2026.03 | 75.2 | — | — | — | — | 59.4 | — | — | — | — | |
| PPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 74.4 | — | — | — | — | 57.3 | — | — | — | — | |
| RLOOType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.06 | 73.9 | — | — | — | — | 52.1 | — | — | — | — | |
| PPOType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.06 | 73.8 | — | — | — | — | 51.5 | — | — | — | — | |
| GRPOType=RL Training, Backbone=Qwen2.5-1.5B-Instruct2026.03 | 73.7 | — | — | — | — | 42.2 | — | — | — | — | |
| RetrospexBase Model=Qwen2.5-7B-Instruct, Learning Paradigm=Offline RL2026.02 | 73.1 | — | — | — | — | 60.4 | — | — | — | — | |
| BaseBackbone=Qwen2.5-1.5B-Instruct, Training Stage=Imitation Learning then Reinforcement Learning initialized by World Modeling2026.06 | 71.1 | — | — | — | — | 59.4 | — | — | — | — | |
| GRPOType=RL Training, Backbone=Qwen2.5-(VL)-7B-Instruct2026.03 | 70.3 | — | — | — | — | 84.8 | — | — | — | — | |
| PPOFramework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.11 | 70.18 | — | — | — | — | 46 | — | — | — | — | |
| GRPOType=RL Training, Backbone=Qwen2.5-(VL)-3B-Instruct2026.03 | 69.5 | — | — | — | — | 53.9 | — | — | — | — | |
| SPEAR+GiGPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 69.2 | — | — | — | — | 44.5 | — | — | — | — | |
| RLOOFramework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.11 | 68.02 | — | — | — | — | 45.1 | — | — | — | — | |
| SimpleMem+GRPOCategory=Memory-Augmented RL-based Methods2026.05 | 67.8 | — | — | — | — | 46.9 | — | — | — | — | |
| GRPOFramework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.11 | 65.83 | — | — | — | — | 44.2 | — | — | — | — | |
| SPEAR+GRPOBase Model=Qwen2.5-1.5B-Instruct2026.07 | 65.4 | — | — | — | — | 46.1 | — | — | — | — | |
| Reinforce++Framework=Agent-R1, Backbone=Qwen3-4B, Type=RL-optimized2025.11 | 63.41 | — | — | — | — | 41.8 | — | — | — | — | |
| ReflexionBase Model=Qwen2.5-7B-Instruct, Learning Paradigm=Non-Parametric2026.02 | 58.1 | — | — | — | — | 28.8 | — | — | — | — | |
| ReflexionCategory=Prompt-based Agentic or Memory-based Methods2026.05 | 58.1 | — | — | — | — | 28.8 | — | — | — | — | |
| Mem0+GRPOCategory=Memory-Augmented RL-based Methods2026.05 | 58.1 | — | — | — | — | 37.5 | — | — | — | — | |
| ReflexionType=Prompting, Backbone=Qwen2.5-7B-Instruct2026.06 | 58.1 | — | — | — | — | 28.8 | — | — | — | — | |
| ReflexionBackbone=Qwen2.5-7B-Instruct, Training Stage=Prompting2026.06 | 58.1 | — | — | — | — | 28.8 | — | — | — | — | |
| ReflexionType=Prompting, Backbone=Qwen2.5-1.5B-Instruct2026.06 | 55.8 | — | — | — | — | 21.9 | — | — | — | — | |
| ReflexionBackbone=Qwen2.5-1.5B-Instruct, Training Stage=Prompting2026.06 | 55.8 | — | — | — | — | 21.9 | — | — | — | — | |
| ReActFramework=Agent-R1, Backbone=Qwen3-4B, Type=Training-free baseline2025.11 | 51.58 | — | — | — | — | 23.8 | — | — | — | — | |
| ReActCategory=Prompt-based Agentic or Memory-based Methods2026.05 | 46.2 | — | — | — | — | 19.5 | — | — | — | — | |
| ReActType=Prompting, Backbone=Qwen2.5-7B-Instruct2026.06 | 46.2 | — | — | — | — | 19.5 | — | — | — | — | |
| ReActBackbone=Qwen2.5-7B-Instruct, Training Stage=Prompting2026.06 | 46.2 | — | — | — | — | 19.5 | — | — | — | — | |
| SkillAdaptorBackbone Model=GPT-5.22026.05 | 44.8 | — | — | — | — | 34.3 | — | — | — | — | |
| SkillAdaptorBackbone Model=GLM-52026.05 | 43.9 | — | — | — | — | 33.3 | — | — | — | — | |
| EvoSkillBackbone Model=GPT-5.22026.05 | 43.1 | — | — | — | — | 33 | — | — | — | — | |
| Gemini-2.5-ProCategory=Closed-source LLMs2026.05 | 42.5 | — | — | — | — | 35.9 | — | — | — | — | |
| EvolveRCategory=Memory-Augmented RL-based Methods2026.05 | 42.5 | — | — | — | — | 17.6 | — | — | — | — | |
| Gemini-2.5-ProType=Prompting, Backbone=Closed-source Model2026.06 | 42.5 | — | — | — | — | 35.9 | — | — | — | — | |
| Gemini-2.5-ProBackbone=Closed-source Models, Training Stage=Prompting2026.06 | 42.5 | — | — | — | — | 35.9 | — | — | — | — | |
| EvoSkillBackbone Model=GLM-52026.05 | 41.6 | — | — | — | — | 32.6 | — | — | — | — | |
| SkillAdaptorBackbone Model=Kimi-K2.52026.05 | 41.6 | — | — | — | — | 33 | — | — | — | — | |
| EvoSkillBackbone Model=Kimi-K2.52026.05 | 40.4 | — | — | — | — | 31.3 | — | — | — | — | |
| ReActType=Prompting, Backbone=Qwen2.5-1.5B-Instruct2026.06 | 40.1 | — | — | — | — | 11.3 | — | — | — | — | |
| ReActBackbone=Qwen2.5-1.5B-Instruct, Training Stage=Prompting2026.06 | 40.1 | — | — | — | — | 11.3 | — | — | — | — | |
| ExpeLBackbone Model=GPT-5.22026.05 | 37 | — | — | — | — | 29.6 | — | — | — | — | |
| AAWMBackbone=Qwen2.5-7B-Instruct, Training Stage=Imitation Learning initialized by World Modeling2026.06 | 37 | — | — | — | — | 16.3 | — | — | — | — | |
| AWMBackbone Model=GPT-5.22026.05 | 36.5 | — | — | — | — | 28.3 | — | — | — | — | |
| Base ModelBackbone Model=GPT-5.22026.05 | 36.2 | — | — | — | — | 25 | — | — | — | — | |
| ExpeLBackbone Model=Kimi-K2.52026.05 | 35.7 | — | — | — | — | 26 | — | — | — | — | |
| ExpeLBackbone Model=GLM-52026.05 | 35.2 | — | — | — | — | 27.3 | — | — | — | — | |
| AWMBackbone Model=GLM-52026.05 | 34.5 | — | — | — | — | 25.6 | — | — | — | — | |
| AWMBackbone Model=Kimi-K2.52026.05 | 34 | — | — | — | — | 25.3 | — | — | — | — | |
| Base ModelBackbone Model=GLM-52026.05 | 33.8 | — | — | — | — | 25.3 | — | — | — | — | |
| A-MemBackbone Model=GPT-5.22026.05 | 33.7 | — | — | — | — | 21.6 | — | — | — | — | |
| SimpleMemCategory=Prompt-based Agentic or Memory-based Methods2026.05 | 33.2 | — | — | — | — | 8.59 | — | — | — | — | |
| Base ModelBackbone Model=Kimi-K2.52026.05 | 32.1 | — | — | — | — | 24.6 | — | — | — | — | |
| GPT-4oCategory=Closed-source LLMs2026.05 | 31.8 | — | — | — | — | 23.7 | — | — | — | — | |
| GPT-4oType=Prompting, Backbone=Closed-source Model2026.06 | 31.8 | — | — | — | — | 23.7 | — | — | — | — | |
| GPT-4oBackbone=Closed-source Models, Training Stage=Prompting2026.06 | 31.8 | — | — | — | — | 23.7 | — | — | — | — | |
| ExpeLCategory=Prompt-based Agentic or Memory-based Methods2026.05 | 30.9 | — | — | — | — | 11.2 | — | — | — | — | |
| A-MemBackbone Model=GLM-52026.05 | 30.9 | — | — | — | — | 20 | — | — | — | — | |
| A-MemBackbone Model=Kimi-K2.52026.05 | 30.3 | — | — | — | — | 22.6 | — | — | — | — | |
| MemRLCategory=Memory-Augmented RL-based Methods2026.05 | 29.5 | — | — | — | — | 9.2 | — | — | — | — |