List your company

Web Agent Task Success on WebArena Shopping 187 tasks

54Task Success RateReasoningBank + MaTTS
37.8842.06546.2550.435Sep 29, 2025

Evaluation results

MethodLinks
ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-pro, k=5, sampling=pass@12025.09545.9
ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-flash, k=5, sampling=pass@12025.09536.3
ReasoningBankBackbone LLM=Gemini-2.5-pro, k=5, sampling=pass@12025.0951.96
ReasoningBankBackbone LLM=Gemini-2.5-flash, k=5, sampling=pass@12025.0949.76.1
AWMBackbone LLM=Gemini-2.5-pro, k=5, sampling=pass@12025.0948.16.4
ReasoningBank + MaTTSBackbone LLM=Claude-3.7-sonnet, k=5, sampling=pass@12025.0947.15.8
SynapseBackbone LLM=Gemini-2.5-pro, k=5, sampling=pass@12025.0946.56.6
No MemoryBackbone LLM=Gemini-2.5-pro, k=5, sampling=pass@12025.0945.57.6
ReasoningBankBackbone LLM=Claude-3.7-sonnet, k=5, sampling=pass@12025.0944.95.6
AWMBackbone LLM=Gemini-2.5-flash, k=5, sampling=pass@12025.0944.47
SynapseBackbone LLM=Gemini-2.5-flash, k=5, sampling=pass@12025.0940.67
SynapseBackbone LLM=Claude-3.7-sonnet, k=5, sampling=pass@12025.0939.65.8
AWMBackbone LLM=Claude-3.7-sonnet, k=5, sampling=pass@12025.0939.67.2
No MemoryBackbone LLM=Gemini-2.5-flash, k=5, sampling=pass@12025.09398.2
No MemoryBackbone LLM=Claude-3.7-sonnet, k=5, sampling=pass@12025.0938.56.1