Tool-use instruction following on ToolBench I2-Category
0.775Pass RateDFSDT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DFSDTBackbone=GPT-4, Retriever=Oracle2023.07 | 0.775 | 63.3 | |
| DFSDTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 0.77 | 58 | |
| ReACTBackbone=GPT-4, Retriever=Oracle2023.07 | 0.72 | 60.3 | |
| DFSDTBackbone=ChatGPT, Retriever=Oracle2023.07 | 0.715 | 64.8 | |
| DFSDTBackbone=ToolLLaMA, Retriever=API Retriever2023.07 | 0.685 | 60.8 | |
| ReACTBackbone=ChatGPT, Retriever=Oracle2023.07 | 0.465 | — | |
| DFSDTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 0.42 | 43.3 | |
| ReACTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 0.315 | 41.8 | |
| DFSDTBackbone=Claude-2, Retriever=Oracle2023.07 | 0.205 | 33.5 | |
| ReACTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 0.145 | 29.8 | |
| ReACTBackbone=Claude-2, Retriever=Oracle2023.07 | 0.06 | 31.5 | |
| ReACT & DFSDTBackbone=Vicuna, Retriever=Oracle2023.07 | 0 | 0 | |
| ReACT & DFSDTBackbone=Alpaca, Retriever=Oracle2023.07 | 0 | 0 |