Web-based Agent Interaction on WebShop (val)
84.4Success RateD2Skill(O3)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| D2Skill(O3)Base Model=Qwen2.5-7B-Instruct2026.03 | 84.4 | 90.1 | |
| D2Skill(120-Steps)Base Model=Qwen3-4B-Instruct-2507 + SFT2026.03 | 81.3 | 89.2 | |
| D2Skill(Gemini-3-Flash)Base Model=Qwen2.5-7B-Instruct2026.03 | 80.5 | 91.1 | |
| GRPO(120-Steps)Base Model=Qwen3-4B-Instruct-2507 + SFT2026.03 | 79.9 | 88.2 | |
| SkillRL(O3)Base Model=Qwen2.5-7B-Instruct2026.03 | 72.7 | 85.2 | |
| GRPOBase Model=Qwen2.5-7B-Instruct2026.03 | 72.6 | 86 | |
| D2Skill(40-Steps)Base Model=Qwen3-4B-Instruct-2507 + SFT2026.03 | 71.9 | 84.1 | |
| GRPO(40-Steps)Base Model=Qwen3-4B-Instruct-2507 + SFT2026.03 | 67.2 | 77.4 | |
| OriginBase Model=Qwen3-4B-Instruct-2507 + SFT2026.03 | 53.1 | 65.6 | |
| SimpleMem+GRPOBase Model=Qwen2.5-7B-Instruct2026.03 | 46.9 | 67.8 | |
| Mem0+GRPOBase Model=Qwen2.5-7B-Instruct2026.03 | 37.5 | 58.1 | |
| GPT-4oBackbone=GPT-4o2026.04 | 36.88 | — | |
| Qwen2-7B + MISEBackbone=Qwen2-7B-Instruct, Technique=MISE2026.04 | 33.68 | — | |
| GPT-4o-miniBackbone=GPT-4o-mini2026.04 | 31.79 | — | |
| LLaMA3-8B + MISEBackbone=LLaMA3-8B-Instruct, Technique=MISE2026.04 | 31.34 | — | |
| LLaMA3-8B + PRMBackbone=LLaMA3-8B-Instruct, Technique=PRM2026.04 | 30.94 | — | |
| Qwen2-7B + PPOBackbone=Qwen2-7B-Instruct, Technique=PPO2026.04 | 30.8 | — | |
| LLaMA3-8B + PPOBackbone=LLaMA3-8B-Instruct, Technique=PPO2026.04 | 30.69 | — | |
| LLaMA3-8B + RFTBackbone=LLaMA3-8B-Instruct, Technique=RFT2026.04 | 30.48 | — | |
| Qwen2-7B + PRMBackbone=Qwen2-7B-Instruct, Technique=PRM2026.04 | 30.43 | — | |
| Qwen2-7BBackbone=Qwen2-7B-Instruct2026.04 | 30.26 | — | |
| LLaMA3-8B + online DPOBackbone=LLaMA3-8B-Instruct, Technique=online DPO2026.04 | 29.31 | — | |
| LLaMA3-8BBackbone=LLaMA3-8B-Instruct2026.04 | 28.9 | — | |
| LLaMA3-8B + ReActBackbone=LLaMA3-8B-Instruct, Technique=ReAct2026.04 | 28.13 | — | |
| Gemini-3-FlashType=Closed-source LLM2026.03 | 16.5 | 14.1 | |
| O3Type=Closed-source LLM2026.03 | 4.7 | 5.8 | |
| OriginBase Model=Qwen2.5-7B-Instruct2026.03 | 3.9 | 16.6 | |
| Gemma2-9BBackbone=Gemma2-9B-Instruct2026.04 | 0 | — | |
| Gemma2-9B + PPOBackbone=Gemma2-9B-Instruct, Technique=PPO2026.04 | 0 | — | |
| Gemma2-9B + PRMBackbone=Gemma2-9B-Instruct, Technique=PRM2026.04 | 0 | — | |
| Gemma2-9B + MISEBackbone=Gemma2-9B-Instruct, Technique=MISE2026.04 | 0 | — |