Tool Invocation on APIBench
79.2HuggingFace AST AccuracyToolTree
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ToolTreeBackbone=GPT-4o, Retriever=BM252026.03 | 79.2 | 74.1 | 66.5 | 73.3 | 2.1 | |
| Tree-of-ThoughtBackbone=GPT-4o, Retriever=BM252026.03 | 78 | 72.4 | 64.8 | 71.7 | 2.5 | |
| ReActBackbone=GPT-4o, Retriever=BM252026.03 | 77.2 | 71 | 63.5 | 70.6 | 5.1 | |
| Zero-shotBackbone=GPT-4o, Retriever=BM252026.03 | 76.5 | 69.8 | 62.3 | 69.5 | 7.8 | |
| ToolTreeBackbone=GPT-4o-mini, Retriever=BM252026.03 | 73.5 | 65.4 | 53.1 | 64 | 7.4 | |
| Tree-of-ThoughtBackbone=GPT-4o-mini, Retriever=BM252026.03 | 71.2 | 62.8 | 49.6 | 61.2 | 9.3 | |
| ReActBackbone=GPT-4o-mini, Retriever=BM252026.03 | 69.8 | 61.5 | 47.2 | 59.5 | 18.5 | |
| Zero-shotBackbone=GPT-4o-mini, Retriever=BM252026.03 | 68.4 | 59.2 | 44.5 | 57.4 | 22.1 |