Tool-use instruction following on ToolBench I1-Instruction
64Pass RateDFSDT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DFSDTBackbone=ToolLLaMA, Retriever=API Retriever2023.07 | 64 | 62.3 | |
| DFSDTBackbone=GPT-4, Retriever=Oracle2023.07 | 60 | 67.5 | |
| DFSDTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 57 | 55 | |
| DFSDTBackbone=ChatGPT, Retriever=Oracle2023.07 | 54.5 | 60.5 | |
| ReACTBackbone=GPT-4, Retriever=Oracle2023.07 | 53.5 | 60 | |
| DFSDTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 43.5 | 40.3 | |
| ReACTBackbone=ChatGPT, Retriever=Oracle2023.07 | 41.5 | — | |
| ReACTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 25 | 45 | |
| DFSDTBackbone=Claude-2, Retriever=Oracle2023.07 | 20.5 | 38 | |
| ReACTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 12 | 28.5 | |
| ReACTBackbone=Claude-2, Retriever=Oracle2023.07 | 5.5 | 31 | |
| ReACT & DFSDTBackbone=Vicuna, Retriever=Oracle2023.07 | 0 | 0 | |
| ReACT & DFSDTBackbone=Alpaca, Retriever=Oracle2023.07 | 0 | 0 |