Tool-use instruction following on ToolBench I1-Tool
71.5Pass RateDFSDT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DFSDTBackbone=GPT-4, Retriever=Oracle2023.07 | 71.5 | 67.8 | |
| DFSDTBackbone=ChatGPT, Retriever=Oracle2023.07 | 65 | 62 | |
| DFSDTBackbone=ToolLLaMA, Retriever=API Retriever2023.07 | 64 | 59 | |
| DFSDTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 61 | 55.3 | |
| ReACTBackbone=GPT-4, Retriever=Oracle2023.07 | 50 | 58.8 | |
| ReACTBackbone=ChatGPT, Retriever=Oracle2023.07 | 44 | — | |
| DFSDTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 44 | 43.8 | |
| DFSDTBackbone=Claude-2, Retriever=Oracle2023.07 | 31 | 44.3 | |
| ReACTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 29 | 42 | |
| ReACTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 20 | 35.3 | |
| ReACTBackbone=Claude-2, Retriever=Oracle2023.07 | 3.5 | 27.8 | |
| ReACT & DFSDTBackbone=Vicuna, Retriever=Oracle2023.07 | 0 | 0 | |
| ReACT & DFSDTBackbone=Alpaca, Retriever=Oracle2023.07 | 0 | 0 |