# E-commerce product search and purchase on WebShop Source: [E-commerce product search and purchase on WebShop]() Collection index: [llms.txt]() Data updated: 2026-07-01T08:44:48.364Z Evaluation of agent performance on e-commerce product search and purchasing tasks using the WebShop dataset, measured by Purchase rate and Strict success metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | E-commerce product search and purchase | | Dataset (including any stated split) | WebShop | | Metric | Strict Success | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Strict Success](): Higher is better; unit: PERCENT. - [Purchase Rate](): Higher is better; unit: PERCENT. - [Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Strict Success | Result date | Report paper | | --- | --- | --- | --- | | SIRI | 81.3 | Not recorded | 6a1e6423dbd7150506f857c1 | | TRIAGE | 77.2 | Not recorded | 6a449f0be0224bd4f4993060 | | TRIAGE | 73.5 | Not recorded | 6a449f0be0224bd4f4993060 | | GiGPO | 72.8 | Not recorded | 6a1e6423dbd7150506f857c1 | | SkillRL | 72.7 | Not recorded | 6a1e6423dbd7150506f857c1 | | GRPO | 70.1 | Not recorded | 6a449f0be0224bd4f4993060 | | PPO | 68.7 | Not recorded | 6a1e6423dbd7150506f857c1 | | GRPO | 66.1 | Not recorded | 6a1e6423dbd7150506f857c1 | | RLOO | 65.7 | Not recorded | 6a1e6423dbd7150506f857c1 | | TRIAGE | 65.4 | Not recorded | 6a449f0be0224bd4f4993060 | | TRIAGE | 55.9 | Not recorded | 6a449f0be0224bd4f4993060 | | TRIAGE | 49.6 | Not recorded | 6a449f0be0224bd4f4993060 | | SimpleMem+GRPO | 46.9 | Not recorded | 6a1e6423dbd7150506f857c1 | | Mem0+GRPO | 37.5 | Not recorded | 6a1e6423dbd7150506f857c1 | | GRPO | 37.5 | Not recorded | 6a449f0be0224bd4f4993060 | | ActFocus | 36.7 | Not recorded | 6a06a87cdbc836b87d829f88 | | TRIAGE | 35.1 | Not recorded | 6a449f0be0224bd4f4993060 | | Reflexion | 28.8 | Not recorded | 6a1e6423dbd7150506f857c1 | | ActFocus | 26.6 | Not recorded | 6a06a87cdbc836b87d829f88 | | Uniform | 22.7 | Not recorded | 6a06a87cdbc836b87d829f88 | | ReAct | 19.5 | Not recorded | 6a1e6423dbd7150506f857c1 | | Evolver | 17.6 | Not recorded | 6a1e6423dbd7150506f857c1 | | Gemini-3-Flash | 16.5 | Not recorded | 6a1e6423dbd7150506f857c1 | | GLM-5.1 | 12.5 | Not recorded | 6a1e6423dbd7150506f857c1 | | Uniform | 9.8 | Not recorded | 6a06a87cdbc836b87d829f88 | | MemRL | 9.2 | Not recorded | 6a1e6423dbd7150506f857c1 | | Origin | 3.9 | Not recorded | 6a1e6423dbd7150506f857c1 | ### SIRI Strict Success: 81.3. Result ID: 6a1ed87dfbc373592c1ae076. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory-Augmented RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### TRIAGE Strict Success: 77.2. Result ID: 6a44d37e73d097d845bfb36e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Judge":"Qwen3-8B-thinking","Evidence":"true"}. This record does not assert independent reproduction. ### TRIAGE Strict Success: 73.5. Result ID: 6a44d37e73d097d845bfb36d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Judge":"Qwen3-8B-thinking","Evidence":"false"}. This record does not assert independent reproduction. ### GiGPO Strict Success: 72.8. Result ID: 6a1ed87dfbc373592c1ae070. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### SkillRL Strict Success: 72.7. Result ID: 6a1ed87dfbc373592c1ae075. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Category":"Memory-Augmented RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GRPO Strict Success: 70.1. Result ID: 6a44d37e73d097d845bfb36b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### PPO Strict Success: 68.7. Result ID: 6a1ed87dfbc373592c1ae06d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GRPO Strict Success: 66.1. Result ID: 6a1ed87dfbc373592c1ae06f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Category":"RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### RLOO Strict Success: 65.7. Result ID: 6a1ed87dfbc373592c1ae06e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### TRIAGE Strict Success: 65.4. Result ID: 6a44d37e73d097d845bfb36c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Judge":"Qwen3-8B no-think"}. This record does not assert independent reproduction. ### TRIAGE Strict Success: 55.9. Result ID: 6a44d37e73d097d845bfb372. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-1.7B-Instruct","Judge":"Qwen3-8B-thinking","Evidence":"true"}. This record does not assert independent reproduction. ### TRIAGE Strict Success: 49.6. Result ID: 6a44d37e73d097d845bfb371. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-1.7B-Instruct","Judge":"Qwen3-8B-thinking","Evidence":"false"}. This record does not assert independent reproduction. ### SimpleMem+GRPO Strict Success: 46.9. Result ID: 6a1ed87dfbc373592c1ae074. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Category":"Memory-Augmented RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Mem0+GRPO Strict Success: 37.5. Result ID: 6a1ed87dfbc373592c1ae073. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory-Augmented RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GRPO Strict Success: 37.5. Result ID: 6a44d37e73d097d845bfb36f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen3-1.7B-Instruct"}. This record does not assert independent reproduction. ### ActFocus Strict Success: 36.7. Result ID: 6a06dc9247f34ce924ee90fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"3B","Evaluation Step":"100"}. This record does not assert independent reproduction. ### TRIAGE Strict Success: 35.1. Result ID: 6a44d37e73d097d845bfb370. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen3-1.7B-Instruct","Judge":"Qwen3-8B no-think"}. This record does not assert independent reproduction. ### Reflexion Strict Success: 28.8. Result ID: 6a1ed87dfbc373592c1ae06c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Category":"Prompt-based Agentic or Memory-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### ActFocus Strict Success: 26.6. Result ID: 6a06dc9247f34ce924ee90fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"1.5B","Evaluation Step":"100"}. This record does not assert independent reproduction. ### Uniform Strict Success: 22.7. Result ID: 6a06dc9247f34ce924ee90f9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"3B","Evaluation Step":"100"}. This record does not assert independent reproduction. ### ReAct Strict Success: 19.5. Result ID: 6a1ed87dfbc373592c1ae06b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Category":"Prompt-based Agentic or Memory-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Evolver Strict Success: 17.6. Result ID: 6a1ed87dfbc373592c1ae072. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory-Augmented RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Gemini-3-Flash Strict Success: 16.5. Result ID: 6a1ed87dfbc373592c1ae068. [Reporting paper]() Recorded evaluation configuration: {"Category":"Closed-source LLMs"}. This record does not assert independent reproduction. ### GLM-5.1 Strict Success: 12.5. Result ID: 6a1ed87dfbc373592c1ae069. [Reporting paper]() Recorded evaluation configuration: {"Category":"Closed-source LLMs"}. This record does not assert independent reproduction. ### Uniform Strict Success: 9.8. Result ID: 6a06dc9247f34ce924ee90fb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"1.5B","Evaluation Step":"100"}. This record does not assert independent reproduction. ### MemRL Strict Success: 9.2. Result ID: 6a1ed87dfbc373592c1ae071. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Category":"Memory-Augmented RL-based Methods","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Origin Strict Success: 3.9. Result ID: 6a1ed87dfbc373592c1ae06a. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.