Web Agent Task Success on WebArena Reddit 106 tasks
83Success Rate (SR)ReasoningBank + MaTTS
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-pro2025.09 | 83 | 5.3 | |
| ReasoningBankBackbone LLM=Gemini-2.5-pro2025.09 | 80.2 | 5.1 | |
| No MemoryBackbone LLM=Gemini-2.5-pro2025.09 | 71.7 | 6 | |
| ReasoningBank + MaTTSBackbone LLM=Gemini-2.5-flash2025.09 | 70.8 | 5.4 | |
| SynapseBackbone LLM=Gemini-2.5-pro2025.09 | 68.9 | 5.9 | |
| AWMBackbone LLM=Gemini-2.5-pro2025.09 | 68.9 | 6.4 | |
| ReasoningBankBackbone LLM=Gemini-2.5-flash2025.09 | 67 | 5.6 | |
| AWMBackbone LLM=Gemini-2.5-flash2025.09 | 62.3 | 6.1 | |
| ReasoningBank + MaTTSBackbone LLM=Claude-3.7-sonnet2025.09 | 60.4 | 5 | |
| SynapseBackbone LLM=Gemini-2.5-flash2025.09 | 59.4 | 6.5 | |
| ReasoningBankBackbone LLM=Claude-3.7-sonnet2025.09 | 57.5 | 5.2 | |
| No MemoryBackbone LLM=Gemini-2.5-flash2025.09 | 55.7 | 6.7 | |
| No MemoryBackbone LLM=Claude-3.7-sonnet2025.09 | 53.8 | 5.5 | |
| SynapseBackbone LLM=Claude-3.7-sonnet2025.09 | 53.8 | 6.1 | |
| AWMBackbone LLM=Claude-3.7-sonnet2025.09 | 52.8 | 7 |