Agentic web navigation and interaction on WebShop (test)
84.38SRRePro
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ReProBackbone=Qwen2.5-7B, Protocol=Best, Category=RePro (Ours)2026.06 | 84.38 | 91.18 | 5.47 | 5.25 | |
| ReProBackbone=Qwen2.5-3B, Protocol=Best, Category=RePro (Ours)2026.06 | 83.59 | 92.09 | 11.71 | 7.68 | |
| ReProBackbone=Qwen2.5-1.5B, Protocol=Best, Category=RePro (Ours)2026.06 | 81.64 | 91.72 | 8.2 | 1.9 | |
| L1Backbone=Qwen2.5-7B, Protocol=Best, Category=Training Methods2026.06 | 80.86 | 91.74 | 1.95 | 5.81 | |
| ReProBackbone=Qwen2.5-7B, Protocol=Avg, Category=RePro (Ours)2026.06 | 80.47 | 89.13 | 5.82 | 4.04 | |
| ReProBackbone=Qwen2.5-3B, Protocol=Avg, Category=RePro (Ours)2026.06 | 79.73 | 90.75 | 11.57 | 6.37 | |
| Meta PromptBackbone=Qwen2.5-7B, Protocol=Best, Category=Training Methods2026.06 | 78.91 | 85.93 | — | — | |
| L1Backbone=Qwen2.5-3B, Protocol=Best, Category=Training Methods2026.06 | 76.56 | 88.53 | 4.68 | 4.12 | |
| L2Backbone=Qwen2.5-3B, Protocol=Best, Category=Training Methods2026.06 | 76.56 | 88.6 | 4.68 | 4.19 | |
| ReProBackbone=Qwen2.5-1.5B, Protocol=Avg, Category=RePro (Ours)2026.06 | 76.13 | 87.56 | 8.98 | 2.29 | |
| L1Backbone=Qwen2.5-7B, Protocol=Avg, Category=Training Methods2026.06 | 75.62 | 87.81 | 0.97 | 2.72 | |
| L2Backbone=Qwen2.5-7B, Protocol=Best, Category=Training Methods2026.06 | 75.39 | 87.54 | -3.52 | 1.61 | |
| L2Backbone=Qwen2.5-1.5B, Protocol=Best, Category=Training Methods2026.06 | 75 | 88.01 | 1.56 | -1.81 | |
| Meta PromptBackbone=Qwen2.5-7B, Protocol=Avg, Category=Training Methods2026.06 | 74.65 | 85.09 | — | — | |
| GiGPOBackbone=Qwen2.5-7B, Protocol=Best, Category=Training Methods2026.06 | 74.22 | 83.98 | -4.69 | -1.95 | |
| Meta PromptBackbone=Qwen2.5-1.5B, Protocol=Best, Category=Training Methods2026.06 | 73.44 | 89.82 | — | — | |
| GRPOBackbone=Qwen2.5-7B, Protocol=Best, Category=Training Methods2026.06 | 73.29 | 82.18 | -5.62 | -3.75 | |
| GiGPOBackbone=Qwen2.5-3B, Protocol=Best, Category=Training Methods2026.06 | 71.88 | 83.94 | 0 | -0.47 | |
| Meta PromptBackbone=Qwen2.5-3B, Protocol=Best, Category=Training Methods2026.06 | 71.88 | 84.41 | — | — | |
| GiGPOBackbone=Qwen2.5-7B, Protocol=Avg, Category=Training Methods2026.06 | 71.76 | 83.77 | -2.89 | -1.32 | |
| L1Backbone=Qwen2.5-3B, Protocol=Avg, Category=Training Methods2026.06 | 71.68 | 86.21 | 3.52 | 1.83 | |
| L2Backbone=Qwen2.5-3B, Protocol=Avg, Category=Training Methods2026.06 | 70.86 | 86.49 | 2.7 | 2.11 | |
| L2Backbone=Qwen2.5-7B, Protocol=Avg, Category=Training Methods2026.06 | 70.59 | 83.45 | -4.06 | -1.64 | |
| L2Backbone=Qwen2.5-1.5B, Protocol=Avg, Category=Training Methods2026.06 | 70.47 | 85.67 | 3.32 | 0.4 | |
| L1Backbone=Qwen2.5-1.5B, Protocol=Best, Category=Training Methods2026.06 | 68.75 | 84.51 | -4.69 | -5.31 | |
| Meta PromptBackbone=Qwen2.5-3B, Protocol=Avg, Category=Training Methods2026.06 | 68.16 | 84.38 | — | — | |
| GiGPOBackbone=Qwen2.5-1.5B, Protocol=Best, Category=Training Methods2026.06 | 67.97 | 81.35 | -5.47 | -8.47 | |
| Meta PromptBackbone=Qwen2.5-1.5B, Protocol=Avg, Category=Training Methods2026.06 | 67.15 | 85.27 | — | — | |
| GRPOBackbone=Qwen2.5-1.5B, Protocol=Best, Category=Training Methods2026.06 | 66.41 | 81.22 | -7.03 | -8.6 | |
| GRPOBackbone=Qwen2.5-7B, Protocol=Avg, Category=Training Methods2026.06 | 64.02 | 76.46 | -10.63 | -8.63 | |
| GiGPOBackbone=Qwen2.5-3B, Protocol=Avg, Category=Training Methods2026.06 | 63.57 | 81.49 | -4.59 | -2.89 | |
| L1Backbone=Qwen2.5-1.5B, Protocol=Avg, Category=Training Methods2026.06 | 63.2 | 82.17 | -3.95 | -3.1 | |
| GRPOBackbone=Qwen2.5-1.5B, Protocol=Avg, Category=Training Methods2026.06 | 62.77 | 77.26 | -4.38 | -8.01 | |
| GiGPOBackbone=Qwen2.5-1.5B, Protocol=Avg, Category=Training Methods2026.06 | 59.84 | 77.68 | -7.31 | -7.59 | |
| GRPOBackbone=Qwen2.5-3B, Protocol=Best, Category=Training Methods2026.06 | 50 | 71.02 | -21.88 | -13.39 | |
| GRPOBackbone=Qwen2.5-3B, Protocol=Avg, Category=Training Methods2026.06 | 44.53 | 63.9 | -23.63 | -20.48 | |
| Retro ProgressBackbone=Qwen2.5-7B, Category=Prompting Methods2026.06 | 23.8 | 51.5 | 9.7 | 0.7 | |
| Meta Prompt for ProgressBackbone=Qwen2.5-3B, Category=Prompting Methods2026.06 | 16.4 | 41.5 | 3.9 | 5 | |
| Random ProgressBackbone=Qwen2.5-3B, Category=Prompting Methods2026.06 | 16.4 | 52.4 | 3.9 | 15.9 | |
| Random Retro ProgressBackbone=Qwen2.5-3B, Category=Prompting Methods2026.06 | 16 | 48.4 | 3.5 | 11.9 | |
| Base PromptBackbone=Qwen2.5-7B, Category=Prompting Methods2026.06 | 14.1 | 50.8 | — | — | |
| Base PromptBackbone=Qwen2.5-3B, Category=Prompting Methods2026.06 | 12.5 | 36.5 | — | — | |
| Random Retro ProgressBackbone=Qwen2.5-7B, Category=Prompting Methods2026.06 | 12.5 | 46.5 | -1.6 | -4.3 | |
| Meta Prompt for ProgressBackbone=Qwen2.5-7B, Category=Prompting Methods2026.06 | 10.9 | 41.4 | -3.2 | -9.4 | |
| Retro ProgressBackbone=Qwen2.5-3B, Category=Prompting Methods2026.06 | 10.6 | 44.5 | -1.9 | 8 | |
| Random ProgressBackbone=Qwen2.5-7B, Category=Prompting Methods2026.06 | 10.2 | 44.4 | -3.9 | -6.4 | |
| Retro ProgressBackbone=Qwen2.5-1.5B, Category=Prompting Methods2026.06 | 5.1 | 20.7 | 4.7 | 15.7 | |
| Random Retro ProgressBackbone=Qwen2.5-1.5B, Category=Prompting Methods2026.06 | 2 | 18.5 | 1.6 | 13.5 | |
| Meta Prompt for ProgressBackbone=Qwen2.5-1.5B, Category=Prompting Methods2026.06 | 1.2 | 3.8 | 0.8 | -1.2 | |
| Random ProgressBackbone=Qwen2.5-1.5B, Category=Prompting Methods2026.06 | 0.8 | 13.2 | 0.4 | 8.2 | |
| Base PromptBackbone=Qwen2.5-1.5B, Category=Prompting Methods2026.06 | 0.4 | 5 | — | — |