# Interactive Decision Making on WebShop (test) Source: [Interactive Decision Making on WebShop (test)]() Collection index: [llms.txt]() Data updated: 2026-05-19T08:19:04.833Z Evaluation of agent performance on interactive decision-making tasks using the test split of the WebShop environment, measured by Score and Success Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Interactive Decision Making | | Dataset (including any stated split) | WebShop (test) | | Metric | Success Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 37 of 37 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Success Rate](): Higher is better; unit: PERCENT. - [Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Success Rate | Result date | Report paper | | --- | --- | --- | --- | | BPO | 97 | Not recorded | 68fa7a631e209131cea82cc5 | | Qwen-2.5-7B-Instruct | 91.5 | Not recorded | 68fa7a631e209131cea82cc5 | | ETO | 90 | Not recorded | 68fa7a631e209131cea82cc5 | | SFT | 88 | Not recorded | 68fa7a631e209131cea82cc5 | | MPO | 87.5 | Not recorded | 68fa7a631e209131cea82cc5 | | GiGPO + PA-MoE | 82.8 | Not recorded | 6997ea6109c33900fcd7645f | | GiGPO + PA-MoE | 82.3 | Not recorded | 6997ea6109c33900fcd7645f | | GEPO | 80.5 | Not recorded | 6997ea6109c33900fcd7645f | | o3-mini | 76.5 | Not recorded | 68fa7a631e209131cea82cc5 | | GEPO | 75.6 | Not recorded | 6997ea6109c33900fcd7645f | | GiGPO w/o std | 75.2 | Not recorded | 6997ea6109c33900fcd7645f | | GiGPO | 72.8 | Not recorded | 6997ea6109c33900fcd7645f | | Qwen-3-8B-Thinking | 71.5 | Not recorded | 68fa7a631e209131cea82cc5 | | PPO + PA-MoE | 70.3 | Not recorded | 6997ea6109c33900fcd7645f | | PPO | 68.7 | Not recorded | 6997ea6109c33900fcd7645f | | GRPO + PA-MoE | 68.5 | Not recorded | 6997ea6109c33900fcd7645f | | RLOO + PA-MoE | 67.8 | Not recorded | 6997ea6109c33900fcd7645f | | GiGPO w/o std | 67.4 | Not recorded | 6997ea6109c33900fcd7645f | | GRPO | 66.1 | Not recorded | 6997ea6109c33900fcd7645f | | RLOO | 65.7 | Not recorded | 6997ea6109c33900fcd7645f | | GiGPO | 65 | Not recorded | 6997ea6109c33900fcd7645f | | Llama-3.1-8B-Instruct | 60 | Not recorded | 68fa7a631e209131cea82cc5 | | RLOO + PA-MoE | 58.6 | Not recorded | 6997ea6109c33900fcd7645f | | Deepseek-R1 | 58.5 | Not recorded | 68fa7a631e209131cea82cc5 | | GRPO + PA-MoE | 57 | Not recorded | 6997ea6109c33900fcd7645f | | GRPO | 56.8 | Not recorded | 6997ea6109c33900fcd7645f | | PPO + PA-MoE | 52.3 | Not recorded | 6997ea6109c33900fcd7645f | | RLOO | 52.1 | Not recorded | 6997ea6109c33900fcd7645f | | PPO | 51.5 | Not recorded | 6997ea6109c33900fcd7645f | | Gemini-2.5-Pro | 35.9 | Not recorded | 6997ea6109c33900fcd7645f | | Reflexion | 28.8 | Not recorded | 6997ea6109c33900fcd7645f | | GPT-4o | 23.7 | Not recorded | 6997ea6109c33900fcd7645f | | Reflexion | 21.9 | Not recorded | 6997ea6109c33900fcd7645f | | ReAct | 19.5 | Not recorded | 6997ea6109c33900fcd7645f | | ReAct | 11.3 | Not recorded | 6997ea6109c33900fcd7645f | | Qwen2.5 | 7.8 | Not recorded | 6997ea6109c33900fcd7645f | | Qwen2.5 | 5.2 | Not recorded | 6997ea6109c33900fcd7645f | ### BPO Success Rate: 97. Result ID: 6a0c1cf3fbc373592c021e42. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach":"System-2","Evaluation Protocol":"Fine-Tuned","Base Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### Qwen-2.5-7B-Instruct Success Rate: 91.5. Result ID: 6a0c1cf3fbc373592c021e3b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach":"System-1","Evaluation Protocol":"Zero-shot","Model Category":"Base Models"}. This record does not assert independent reproduction. ### ETO Success Rate: 90. Result ID: 6a0c1cf3fbc373592c021e40. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach":"System-2","Evaluation Protocol":"Fine-Tuned","Base Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### SFT Success Rate: 88. Result ID: 6a0c1cf3fbc373592c021e3f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach":"System-2","Evaluation Protocol":"Fine-Tuned","Base Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### MPO Success Rate: 87.5. Result ID: 6a0c1cf3fbc373592c021e41. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach":"System-2","Evaluation Protocol":"Fine-Tuned","Base Model":"Llama-3.1-8B-Instruct"}. This record does not assert independent reproduction. ### GiGPO + PA-MoE Success Rate: 82.8. Result ID: 699b04245698665ec496561b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GiGPO + PA-MoE Success Rate: 82.3. Result ID: 699b04245698665ec496560e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### GEPO Success Rate: 80.5. Result ID: 699b04245698665ec4965612. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### o3-mini Success Rate: 76.5. Result ID: 6a0c1cf3fbc373592c021e3c. [Reporting paper]() Recorded evaluation configuration: {"Approach":"System-2","Evaluation Protocol":"Zero-shot","Model Category":"Reasoning LLMs"}. This record does not assert independent reproduction. ### GEPO Success Rate: 75.6. Result ID: 699b04245698665ec4965605. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### GiGPO w/o std Success Rate: 75.2. Result ID: 699b04245698665ec496561a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GiGPO Success Rate: 72.8. Result ID: 699b04245698665ec4965619. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Qwen-3-8B-Thinking Success Rate: 71.5. Result ID: 6a0c1cf3fbc373592c021e3e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach":"System-2","Evaluation Protocol":"Zero-shot","Model Category":"Reasoning LLMs"}. This record does not assert independent reproduction. ### PPO + PA-MoE Success Rate: 70.3. Result ID: 699b04245698665ec4965614. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### PPO Success Rate: 68.7. Result ID: 699b04245698665ec4965613. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GRPO + PA-MoE Success Rate: 68.5. Result ID: 699b04245698665ec4965618. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### RLOO + PA-MoE Success Rate: 67.8. Result ID: 699b04245698665ec4965616. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GiGPO w/o std Success Rate: 67.4. Result ID: 699b04245698665ec496560d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### GRPO Success Rate: 66.1. Result ID: 699b04245698665ec4965617. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### RLOO Success Rate: 65.7. Result ID: 699b04245698665ec4965615. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GiGPO Success Rate: 65. Result ID: 699b04245698665ec496560c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### Llama-3.1-8B-Instruct Success Rate: 60. Result ID: 6a0c1cf3fbc373592c021e3a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Approach":"System-1","Evaluation Protocol":"Zero-shot","Model Category":"Base Models"}. This record does not assert independent reproduction. ### RLOO + PA-MoE Success Rate: 58.6. Result ID: 699b04245698665ec4965609. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### Deepseek-R1 Success Rate: 58.5. Result ID: 6a0c1cf3fbc373592c021e3d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Approach":"System-2","Evaluation Protocol":"Zero-shot","Model Category":"Reasoning LLMs"}. This record does not assert independent reproduction. ### GRPO + PA-MoE Success Rate: 57. Result ID: 699b04245698665ec496560b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### GRPO Success Rate: 56.8. Result ID: 699b04245698665ec496560a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### PPO + PA-MoE Success Rate: 52.3. Result ID: 699b04245698665ec4965607. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### RLOO Success Rate: 52.1. Result ID: 699b04245698665ec4965608. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### PPO Success Rate: 51.5. Result ID: 699b04245698665ec4965606. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"RL Training","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### Gemini-2.5-Pro Success Rate: 35.9. Result ID: 699b04245698665ec4965601. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Closed-Source Model"}. This record does not assert independent reproduction. ### Reflexion Success Rate: 28.8. Result ID: 699b04245698665ec4965611. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### GPT-4o Success Rate: 23.7. Result ID: 699b04245698665ec4965600. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Closed-Source Model"}. This record does not assert independent reproduction. ### Reflexion Success Rate: 21.9. Result ID: 699b04245698665ec4965604. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### ReAct Success Rate: 19.5. Result ID: 699b04245698665ec4965610. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### ReAct Success Rate: 11.3. Result ID: 699b04245698665ec4965603. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. ### Qwen2.5 Success Rate: 7.8. Result ID: 699b04245698665ec496560f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### Qwen2.5 Success Rate: 5.2. Result ID: 699b04245698665ec4965602. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Type":"Prompting","Backbone":"Qwen2.5-1.5B-Instruct"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.