Tool Planning on RestBench Spotify
61.25Pass RateLATS
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| LATSModel=GPT-4o2026.03 | 61.25 | 75.8 | 68.53 | |
| ToolTreeModel=GPT-4o2026.03 | 60.87 | 78.84 | 71.36 | |
| DFSDTModel=GPT-4o2026.03 | 55.48 | 71.63 | 63.55 | |
| ReActModel=GPT-4o2026.03 | 53.27 | 60.72 | 57 | |
| Zero-shotModel=GPT-4o2026.03 | 49.54 | 50 | 49.77 | |
| CoTModel=GPT-4o2026.03 | 47.92 | 44.55 | 46.23 | |
| ToolTreeModel=GPT-4o-mini2026.03 | 42.08 | 72.18 | 57.74 | |
| LATSModel=GPT-4o-mini2026.03 | 39.81 | 72.85 | 56.33 | |
| DFSDTModel=GPT-4o-mini2026.03 | 35.1 | 65.47 | 50.28 | |
| ReActModel=GPT-4o-mini2026.03 | 32.64 | 59.95 | 46.3 | |
| CoTModel=GPT-4o-mini2026.03 | 29.82 | 53.1 | 41.46 | |
| Zero-shotModel=GPT-4o-mini2026.03 | 26.44 | 50 | 38.22 |