Web Agent Task Success on WebArena 182 tasks (Admin)
58.2Success Rate (SR)ReasoningBank + MaTTS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-pro2025.09 | 58.2 | 7.4 | |
| ReasoningBankBackbone LLM=Gemini-2.5-pro2025.09 | 56.6 | 7.7 | |
| ReasoningBank + MaTTSBackbone LLM=Claude-3.7-sonnet2025.09 | 55.5 | 7.4 | |
| ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-flash2025.09 | 53.8 | 7.6 | |
| ReasoningBankBackbone LLM=Claude-3.7-sonnet2025.09 | 53.3 | 7.6 | |
| SynapseBackbone LLM=Gemini-2.5-pro2025.09 | 52.2 | 8.9 | |
| ReasoningBankBackbone LLM=Gemini-2.5-flash2025.09 | 51.1 | 8.2 | |
| No MemoryBackbone LLM=Gemini-2.5-pro2025.09 | 51.1 | 8.7 | |
| SynapseBackbone LLM=Claude-3.7-sonnet2025.09 | 50.5 | 8.5 | |
| No MemoryBackbone LLM=Claude-3.7-sonnet2025.09 | 49.5 | 8.4 | |
| AWMBackbone LLM=Gemini-2.5-pro2025.09 | 49.3 | 9.8 | |
| AWMBackbone LLM=Claude-3.7-sonnet2025.09 | 47.8 | 9.3 | |
| AWMBackbone LLM=Gemini-2.5-flash2025.09 | 46.7 | 8.8 | |
| SynapseBackbone LLM=Gemini-2.5-flash2025.09 | 45.1 | 9.1 | |
| No MemoryBackbone LLM=Gemini-2.5-flash2025.09 | 44.5 | 9.5 |