Shopping Agent Task on Amazon Hard
45Kimi K2 Success RateSTRUCTUREDAGENT
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| STRUCTUREDAGENTBackbone=Claude 3.5 Sonnet2026.03 | 45 | 28.3 | 33.3 | 35.6 | 58.3 | |
| StructuredAgentMemBackbone=Claude 3.5 Sonnet2026.03 | 43.3 | 28.3 | 41.7 | 37.8 | 63.3 | |
| AgentOccamBackbone=Claude 3.5 Sonnet2026.03 | 36.7 | 16.7 | 31.7 | 28.4 | 56.3 | |
| BasicClaudeActionBackbone=Claude 3.5 Sonnet2026.03 | 28.3 | 18.3 | 23.3 | 23.3 | 0.4 | |
| StePBackbone=Claude 3.5 Sonnet2026.03 | 13.3 | 10 | 13.3 | 12.2 | — |