Web agent task completion on WebArena (test)
42.9Shopping Success RateAPEX
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| APEXevaluation_subset=Final-3 episodes, episodes_per_task=52026.05 | 42.9 | 55.1 | 68.2 | 60.2 | 57.6 | 55.9 | — | |
| GPT-5.1 + PageDigestBackbone=GPT-5.1, PageDigest=true2026.05 | 41.1 | 54.5 | 60.5 | 50.5 | 28.4 | 47.5 | 4,302 | |
| Memoryevaluation_subset=Final-3 episodes, episodes_per_task=52026.05 | 39.1 | 53.2 | 53.8 | 53.9 | 55.2 | 50.5 | — | |
| GPT-5.1Backbone=GPT-5.1, PageDigest=false2026.05 | 39 | 57.1 | 65.1 | 46.1 | 24.8 | 45.2 | 6,116 | |
| EvoTestevaluation_subset=Final-3 episodes, episodes_per_task=52026.05 | 38.9 | 53.8 | 50.9 | 54.4 | 52.9 | 49.9 | — | |
| Qwen3.5-27B + PageDigestBackbone=Qwen3.5-27B, PageDigest=true2026.05 | 38.6 | 46.2 | 60.4 | 35.2 | 18.9 | 39.9 | 2,654 | |
| ACEevaluation_subset=Final-3 episodes, episodes_per_task=52026.05 | 35.9 | 52.4 | 52.5 | 51.6 | 55.7 | 48.6 | — | |
| SteP + PageDigestBackbone=GPT-4o, PageDigest=true2026.05 | 35.8 | 37.1 | 63.2 | 50 | 15.4 | 38.7 | 3,693 | |
| AgentOccam + PageDigestBackbone=GPT-4o, PageDigest=true2026.05 | 35.6 | 40 | 68.4 | 50 | 23.1 | 41.3 | 3,365 | |
| Gemini 3.1 Flash-Lite + PageDigestBackbone=Gemini 3.1 Flash-Lite, PageDigest=true2026.05 | 34.9 | 44.5 | 59.3 | 42.6 | 28.4 | 41.6 | 3,207 | |
| Staticevaluation_subset=Final-3 episodes, episodes_per_task=52026.05 | 34.7 | 44.8 | 45.7 | 36.3 | 56.3 | 42.3 | — | |
| Reflexionevaluation_subset=Final-3 episodes, episodes_per_task=52026.05 | 34.1 | 46.7 | 36.2 | 50.8 | 54.2 | 44.3 | — | |
| Gemini 3.1 Flash-LiteBackbone=Gemini 3.1 Flash-Lite, PageDigest=false2026.05 | 33.3 | 41.2 | 54 | 44.7 | 22.9 | 39.4 | 6,705 | |
| DeepSeek-V3.2 + PageDigestBackbone=DeepSeek-V3.2, PageDigest=true2026.05 | 33.3 | 53.7 | 61.1 | 46.5 | 21.1 | 43.4 | 4,521 | |
| StePBackbone=GPT-4o, PageDigest=false2026.05 | 32.6 | 45.7 | 71.4 | 46.9 | 12.8 | 39.5 | 7,136 | |
| DeepSeek-V3.2Backbone=DeepSeek-V3.2, PageDigest=false2026.05 | 30.7 | 51.1 | 58.5 | 40.4 | 21.1 | 40.4 | 7,158 | |
| AgentOccamBackbone=GPT-4o, PageDigest=false2026.05 | 26.7 | 36.3 | 73.7 | 66.7 | 11.5 | 40.6 | 4,025 | |
| Qwen3.5-27BBackbone=Qwen3.5-27B, PageDigest=false2026.05 | 22.9 | 53.8 | 58.8 | 35.3 | 22.9 | 38.2 | 5,767 |