Interactive Agent Task on WebShop
14Efficiency MultiplierACON
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ACONBackbone=gpt-4o-mini, Per-step LLM call=true2026.05 | 14 | 0.0021 | |
| ACONBackbone=qwen3.5-flash, Per-step LLM call=true2026.05 | 9.3 | 0.0008 | |
| AGORABackbone=gpt-4o-mini, Per-step LLM call=false2026.05 | 8.9 | 0.0115 | |
| HiAgentBackbone=gpt-4o-mini, Per-step LLM call=true2026.05 | 8.3 | 0.0032 | |
| AgentDietBackbone=gpt-4o-mini, Per-step LLM call=true2026.05 | 7.9 | 0.0034 | |
| AgentDietBackbone=qwen3.5-flash, Per-step LLM call=true2026.05 | 7.7 | 0.0008 | |
| HiAgentBackbone=qwen3.5-flash, Per-step LLM call=true2026.05 | 5.9 | 0.001 | |
| AgentDietBackbone=gpt-5-mini, Per-step LLM call=true2026.05 | 5.4 | 0.0063 | |
| ObsMaskBackbone=gpt-4o-mini, Per-step LLM call=false2026.05 | 4.8 | 0.0053 | |
| AGORABackbone=qwen3.5-flash, Per-step LLM call=false2026.05 | 4.4 | 0.0026 | |
| ACONBackbone=gpt-5-mini, Per-step LLM call=true2026.05 | 4.4 | 0.011 | |
| HiAgentBackbone=gpt-5-mini, Per-step LLM call=true2026.05 | 3.8 | 0.0084 | |
| AGORABackbone=gpt-5-mini, Per-step LLM call=false2026.05 | 3.3 | 0.0184 | |
| ObsMaskBackbone=qwen3.5-flash, Per-step LLM call=false2026.05 | 2.7 | 0.0018 | |
| ObsMaskBackbone=gpt-5-mini, Per-step LLM call=false2026.05 | 2.2 | 0.0118 |