Web Agent Task Success on WebArena Overall 684 tasks
56.3Task Success Rate (SR)ReasoningBank + MaTTS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-pro2025.09 | 56.3 | 7.1 | |
| ReasoningBankBackbone LLM=Gemini-2.5-pro2025.09 | 53.9 | 7.4 | |
| ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-flash2025.09 | 51.8 | 7.9 | |
| ReasoningBankBackbone LLM=Gemini-2.5-flash2025.09 | 48.8 | 8.3 | |
| ReasoningBank + MaTTSBackbone LLM=Claude-3.7-sonnet2025.09 | 48.8 | 7.2 | |
| SynapseBackbone LLM=Gemini-2.5-pro2025.09 | 47.7 | 8.5 | |
| AWMBackbone LLM=Gemini-2.5-pro2025.09 | 47.6 | 8.7 | |
| No MemoryBackbone LLM=Gemini-2.5-pro2025.09 | 46.7 | 8.8 | |
| ReasoningBankBackbone LLM=Claude-3.7-sonnet2025.09 | 46.3 | 7.3 | |
| AWMBackbone LLM=Gemini-2.5-flash2025.09 | 44.1 | 9 | |
| SynapseBackbone LLM=Claude-3.7-sonnet2025.09 | 42.6 | 7.9 | |
| SynapseBackbone LLM=Gemini-2.5-flash2025.09 | 42.1 | 9.2 | |
| No MemoryBackbone LLM=Claude-3.7-sonnet2025.09 | 41.7 | 8 | |
| AWMBackbone LLM=Claude-3.7-sonnet2025.09 | 40.8 | 8.9 | |
| No MemoryBackbone LLM=Gemini-2.5-flash2025.09 | 40.5 | 9.7 |