Task Success on WebShop
61Real Success ScoreClaude-sonnet-4.5
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Claude-sonnet-4.5World Model=Qwen2.5-7B WorldModel2025.12 | 61 | 49 | 50 | 0.82 | |
| Claude-sonnet-4.5World Model=Llama3.1-8B WorldModel2025.12 | 61 | 60 | 55 | 0.9 | |
| GPT-5World Model=Qwen2.5-7B WorldModel2025.12 | 51 | 33.03 | 31.19 | 0.61 | |
| GPT-5World Model=Llama3.1-8B WorldModel2025.12 | 51 | 31.19 | 30.28 | 0.59 | |
| GPT-4oWorld Model=Qwen2.5-7B WorldModel2025.12 | 29.36 | 17.43 | 16.51 | 0.56 | |
| GPT-4oWorld Model=Llama3.1-8B WorldModel2025.12 | 29.36 | 23.81 | 22.62 | 0.77 | |
| Gemini-2.5-flashWorld Model=Qwen2.5-7B WorldModel2025.12 | 25 | 21.1 | 18.35 | 0.73 | |
| Gemini-2.5-flashWorld Model=Llama3.1-8B WorldModel2025.12 | 25 | 22.02 | 17.43 | 0.7 | |
| GPT-4.1World Model=Qwen2.5-7B WorldModel2025.12 | 21.14 | 12.22 | 12.22 | 0.58 | |
| GPT-4.1World Model=Llama3.1-8B WorldModel2025.12 | 21.14 | 19.27 | 17.43 | 0.82 | |
| GPT-4-turboWorld Model=Qwen2.5-7B WorldModel2025.12 | 17.73 | 14.89 | 11.7 | 0.66 | |
| GPT-4-turboWorld Model=Llama3.1-8B WorldModel2025.12 | 17.73 | 25.47 | 17.92 | 1.01 | |
| GPT-4o-miniWorld Model=Qwen2.5-7B WorldModel2025.12 | 5.99 | 4.85 | 0.97 | 0.16 | |
| GPT-4o-miniWorld Model=Llama3.1-8B WorldModel2025.12 | 5.99 | 2.75 | 0 | 0 |