Plan-level Tool Use on τ2-retail (test)
76.43Overall Success RateFlowAgent
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FlowAgentExecutor=Qwen2.5-7B-Instruct2026.05 | 76.43 | 57.14 | 77.45 | 58.92 | 96.19 | 59.29 | |
| FlowAgentExecutor=Llama-3.2-3B-Instruct2026.05 | 75.71 | 55.84 | 77.14 | 58.44 | 95.24 | 55.73 | |
| Standard SFTExecutor=Qwen2.5-7B-Instruct2026.05 | 65.71 | 37.66 | 67.14 | 40.26 | 92.68 | 43.87 | |
| Standard SFTExecutor=Llama-3.2-3B-Instruct2026.05 | 62.14 | 31.17 | 62.86 | 32.47 | 93.18 | 43.48 | |
| RAG PlanningExecutor=Llama-3.2-3B-Instruct2026.05 | 52.86 | 50.65 | 55 | 54.55 | 90.91 | 32.41 | |
| RAG PlanningExecutor=Qwen2.5-7B-Instruct2026.05 | 52.14 | 49.35 | 52.14 | 49.35 | 100 | 34.78 | |
| ToolRLExecutor=Qwen2.5-7B-Instruct2026.05 | 47.86 | 50.65 | 49.29 | 53.25 | 96.43 | 52.57 | |
| Direct-LLMExecutor=Llama-3.2-3B-Instruct2026.05 | 20 | 7.79 | 20.71 | 9.09 | 80.6 | 12.25 | |
| ToolRLExecutor=Llama-3.2-3B-Instruct2026.05 | 13.57 | 11.69 | 15 | 14.29 | 93.62 | 10.28 | |
| Direct-LLMExecutor=Qwen2.5-7B-Instruct2026.05 | 12.14 | 22.08 | 12.86 | 23.38 | 94.12 | 43.08 |