E-commerce Interaction on WebShop 128 tasks (test)
89.4ScoreSDAR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SDARBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 89.4 | 82.8 | |
| RLSDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 87.4 | 77.3 | |
| Skill-GRPOBackbone=Qwen2.5-7B-Instruct, Validation with skills=true2026.06 | 87 | 81.2 | |
| GRPO+OPSDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 86.8 | 76.5 | |
| Skill-SDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 86.1 | 76.5 | |
| OPIDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 85.3 | 79.7 | |
| SDARBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 85 | 68 | |
| OPIDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 85 | 74.2 | |
| RLSDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 84.4 | 66.4 | |
| Skill-SDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 81.8 | 53.9 | |
| GRPOBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 80.9 | 72.6 | |
| Skill-GRPOBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 80.4 | 71.9 | |
| Skill-GRPOBackbone=Qwen3-1.7B-Instruct, Validation with skills=true2026.06 | 80.4 | 50 | |
| GRPOBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 79.8 | 63.3 | |
| OPIDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 79.6 | 64.8 | |
| GRPO+OPSDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 77.8 | 66.4 | |
| Skill-GRPOBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 77.3 | 60.9 | |
| SDARBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 76.8 | 58.6 | |
| Skill-GRPOBackbone=Qwen2.5-3B-Instruct, Validation with skills=true2026.06 | 76.3 | 66.4 | |
| Skill-SDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 75.9 | 64 | |
| RLSDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 74 | 50.8 | |
| Skill-GRPOBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 73.4 | 46.1 | |
| GRPO+OPSDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 70.7 | 38.3 | |
| GRPOBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 67.3 | 38.3 | |
| OPSDBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 47.4 | 9.3 | |
| VanillaBackbone=Qwen3-1.7B-Instruct, Validation with skills=false2026.06 | 46.5 | 4.7 | |
| Skill-PromptBackbone=Qwen3-1.7B-Instruct, Validation with skills=true2026.06 | 23 | 2.3 | |
| OPSDBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 11.3 | 3.1 | |
| VanillaBackbone=Qwen2.5-3B-Instruct, Validation with skills=false2026.06 | 6.7 | 0.8 | |
| VanillaBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 5.9 | 1.6 | |
| OPSDBackbone=Qwen2.5-7B-Instruct, Validation with skills=false2026.06 | 4.5 | 2.3 | |
| Skill-PromptBackbone=Qwen2.5-7B-Instruct, Validation with skills=true2026.06 | 1.7 | 0.8 | |
| Skill-PromptBackbone=Qwen2.5-3B-Instruct, Validation with skills=true2026.06 | 0.2 | 0.8 |