Tool Planning on ToolBench (EM, ELR, ACPL)
42.6EM (%)GRAFT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GRAFTLLM=Mistral-7B2026.05 | 42.6 | 40 | 1.35 | |
| GRAFTLLM=Llama-3.2-3B2026.05 | 41.8 | 36 | 1.33 | |
| GRAFTLLM=Qwen2.5-1.5B2026.05 | 39.4 | 35 | 1.28 | |
| ToolGenLLM=Mistral-7B2026.05 | 35 | 33 | 1.21 | |
| ToolGenLLM=Llama-3.2-3B2026.05 | 33 | 31 | 1.18 | |
| ToolGenLLM=Qwen2.5-1.5B2026.05 | 31.4 | 28 | 1.03 | |
| PLaGLLM=Mistral-7B2026.05 | 21.95 | 39 | 0.9 | |
| PLaGLLM=Llama-3.2-3B2026.05 | 16.86 | 33 | 0.67 | |
| HuggingGPTLLM=Mistral-7B2026.05 | 16.01 | 38 | 0.6 | |
| DirectLLM=Llama-3.2-3B2026.05 | 15.58 | 22 | 0.57 | |
| DirectLLM=Mistral-7B2026.05 | 15.47 | 37 | 0.52 | |
| PLaGLLM=Qwen2.5-1.5B2026.05 | 15.29 | 34 | 0.65 | |
| HuggingGPTLLM=Llama-3.2-3B2026.05 | 15.15 | 25 | 0.57 | |
| GToolLLM=Mistral-7B2026.05 | 13.8 | 24 | 0.58 | |
| BeamSearchLLM=Mistral-7B2026.05 | 13.26 | 51 | 0.57 | |
| GToolLLM=Llama-3.2-3B2026.05 | 11.8 | 22 | 0.52 | |
| BeamSearchLLM=Llama-3.2-3B2026.05 | 11.7 | 74 | 0.57 | |
| GToolLLM=Qwen2.5-1.5B2026.05 | 10.8 | 21 | 0.51 | |
| BeamSearchLLM=Qwen2.5-1.5B2026.05 | 10.59 | 67 | 0.51 | |
| HuggingGPTLLM=Qwen2.5-1.5B2026.05 | 9.21 | 32 | 0.36 | |
| DirectLLM=Qwen2.5-1.5B2026.05 | 9.15 | 31 | 0.32 | |
| GNN4PlanLLM=Mistral-7B2026.05 | 6.7 | 100 | 0.72 | |
| GNN4PlanLLM=Llama-3.2-3B2026.05 | 6.61 | 100 | 0.7 | |
| GNN4PlanLLM=Qwen2.5-1.5B2026.05 | 6.4 | 100 | 0.7 |