Web Agent on WebArena
35.8Success Ratew/o Attack
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| w/o AttackArchitecture=Hier.2026.05 | 35.8 | — | — | |
| w/o AttackArchitecture=Flat2026.05 | 33.3 | — | — | |
| Prompt InfectionArchitecture=Hier.2026.05 | 31.9 | — | — | |
| Forced OutputArchitecture=Hier.2026.05 | 31.7 | — | — | |
| w/o AttackArchitecture=Chain2026.05 | 30.8 | — | — | |
| InjecAgentArchitecture=Hier.2026.05 | 30.2 | — | — | |
| Qwen2.5-VL Agent + VisCriticbase_agent=Qwen2.5-VL Agent, critic_module=VisCritic2026.06 | 29.4 | — | — | |
| Forced OutputArchitecture=Flat2026.05 | 27.2 | — | — | |
| Qwen2.5-VL Agent + GUI-Critic-R1base_agent=Qwen2.5-VL Agent, critic_module=GUI-Critic-R12026.06 | 26.5 | — | — | |
| Forced OutputArchitecture=Chain2026.05 | 26.1 | — | — | |
| Qwen2.5-VL Agent + GuidNavbase_agent=Qwen2.5-VL Agent, critic_module=GuidNav2026.06 | 26.1 | — | — | |
| Qwen2.5-VL Agent + GUI-PRAbase_agent=Qwen2.5-VL Agent, critic_module=GUI-PRA2026.06 | 24.9 | — | — | |
| Prompt InfectionArchitecture=Chain2026.05 | 24.3 | — | — | |
| InjecAgentArchitecture=Flat2026.05 | 24.1 | — | — | |
| Web FraudArchitecture=Hier.2026.05 | 23.8 | — | — | |
| InjecAgentArchitecture=Chain2026.05 | 23.5 | — | — | |
| ShowUI + VisCriticbase_agent=ShowUI, critic_module=VisCritic2026.06 | 23.2 | — | — | |
| Qwen2.5-VL Agentbase_agent=Qwen2.5-VL Agent2026.06 | 22.8 | — | — | |
| Prompt InfectionArchitecture=Flat2026.05 | 22.7 | — | — | |
| single-stageretrieval=single-stage2026.06 | 20.45 | 20.2614 | 0.0284 | |
| rerank-FFNreranker=FFN2026.06 | 20.45 | 19.9091 | 0.0398 | |
| rerank-SVMRankreranker=SVMRank2026.06 | 20.45 | 20.2841 | 0.017 | |
| ShowUI + GUI-PRAbase_agent=ShowUI, critic_module=GUI-PRA2026.06 | 19.7 | — | — | |
| Web FraudArchitecture=Flat2026.05 | 19.6 | — | — | |
| WEASEL (10K steps)Model=Qwen3-8B, # Data=10K, # Epochs=2, Time=8.3 hr, Speed-up=10.7×2026.05 | 19.2 | — | — | |
| Evo-AttackerArchitecture=Hier.2026.05 | 18.4 | — | — | |
| CogAgent + VisCriticbase_agent=CogAgent, critic_module=VisCritic2026.06 | 18.4 | — | — | |
| Full (52K steps)Model=Qwen3-8B, # Data=52K, # Epochs=2, Time=88.5 hr, Speed-up=1.0×2026.05 | 18.2 | — | — | |
| no retrievalretrieval=none2026.06 | 18.18 | 21.9886 | -0.0511 | |
| rerank-LambdaMARTreranker=LambdaMART2026.06 | 18.18 | 20.4205 | -0.0341 | |
| Web FraudArchitecture=Chain2026.05 | 18.1 | — | — | |
| Qwen3-8BModel=Qwen3-8B2026.05 | 18 | — | — | |
| ShowUIbase_agent=ShowUI2026.06 | 17.6 | — | — | |
| Pruning + Sampling (10K steps)Model=Qwen3-8B, # Data=10K, # Epochs=2, Time=7.0 hr, Speed-up=12.6×2026.05 | 17.5 | — | — | |
| Pruning + LLM-Judge (10K steps)Model=Qwen3-8B, # Data=10K, # Epochs=2, Time=7.0 hr, Speed-up=12.6×2026.05 | 16.6 | — | — | |
| SeeClick + VisCriticbase_agent=SeeClick, critic_module=VisCritic2026.06 | 16.4 | — | — | |
| SeeClick + GUI-Critic-R1base_agent=SeeClick, critic_module=GUI-Critic-R12026.06 | 15.7 | — | — | |
| SeeClick + BacktrackAgentbase_agent=SeeClick, critic_module=BacktrackAgent2026.06 | 15.3 | — | — | |
| CogAgentbase_agent=CogAgent2026.06 | 15.1 | — | — | |
| SeeClick + GUI-PRAbase_agent=SeeClick, critic_module=GUI-PRA2026.06 | 14.6 | — | — | |
| Evo-AttackerArchitecture=Flat2026.05 | 14.5 | — | — | |
| Evo-AttackerArchitecture=Chain2026.05 | 13.4 | — | — | |
| Pruning (52K steps)Model=Qwen3-8B, # Data=52K, # Epochs=2, Time=44.0 hr, Speed-up=2.0×2026.05 | 12.7 | — | — | |
| SeeClickbase_agent=SeeClick2026.06 | 12.4 | — | — | |
| WEASEL (10K steps)Model=Qwen2.5-7B-Instruct, # Data=10K, # Epochs=4, Time=12.0 hr, Speed-up=11.3×2026.05 | 9.5 | — | — | |
| Full (52K steps)Model=Qwen2.5-7B-Instruct, # Data=52K, # Epochs=4, Time=136.0 hr, Speed-up=1.0×2026.05 | 8.7 | — | — | |
| Pruning (52K steps)Model=Qwen2.5-7B-Instruct, # Data=52K, # Epochs=4, Time=62.0 hr, Speed-up=2.2×2026.05 | 8.6 | — | — | |
| Pruning + Sampling (10K steps)Model=Qwen2.5-7B-Instruct, # Data=10K, # Epochs=4, Time=12.0 hr, Speed-up=11.3×2026.05 | 8.1 | — | — | |
| Pruning + LLM-Judge (10K steps)Model=Qwen2.5-7B-Instruct, # Data=10K, # Epochs=4, Time=12.0 hr, Speed-up=11.3×2026.05 | 7.8 | — | — | |
| WEASEL (10K steps)Model=Gemma3-4B-IT, # Data=10K, # Epochs=2, Time=6.4 hr, Speed-up=12.5×2026.05 | 5.5 | — | — | |
| Qwen2.5-7B-InstructModel=Qwen2.5-7B-Instruct2026.05 | 5.2 | — | — | |
| Pruning + LLM-Judge (10K steps)Model=Gemma3-4B-IT, # Data=10K, # Epochs=2, Time=6.4 hr, Speed-up=12.5×2026.05 | 5.2 | — | — | |
| Pruning (52K steps)Model=Gemma3-4B-IT, # Data=52K, # Epochs=2, Time=30.0 hr, Speed-up=2.7×2026.05 | 4.8 | — | — | |
| Pruning + Sampling (10K steps)Model=Gemma3-4B-IT, # Data=10K, # Epochs=2, Time=6.4 hr, Speed-up=12.5×2026.05 | 4.7 | — | — | |
| Full (52K steps)Model=Gemma3-4B-IT, # Data=52K, # Epochs=2, Time=80.0 hr, Speed-up=1.0×2026.05 | 4.3 | — | — | |
| Gemma3-4B-ITModel=Gemma3-4B-IT2026.05 | 2.7 | — | — |