Tool-use instruction following on ToolBench I2-Instruction
81.5Pass RateDFSDT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DFSDTBackbone=ToolLLaMA, Retriever=API Retriever2023.07 | 81.5 | 68.5 | |
| DFSDTBackbone=GPT-4, Retriever=Oracle2023.07 | 79.5 | 73.3 | |
| DFSDTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 77 | 68.5 | |
| DFSDTBackbone=ChatGPT, Retriever=Oracle2023.07 | 75 | 72 | |
| ReACTBackbone=GPT-4, Retriever=Oracle2023.07 | 67 | 65.8 | |
| ReACTBackbone=ChatGPT, Retriever=Oracle2023.07 | 42.5 | — | |
| DFSDTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 37 | 40.5 | |
| ReACTBackbone=ToolLLaMA, Retriever=Oracle2023.07 | 30.5 | 50.8 | |
| DFSDTBackbone=Claude-2, Retriever=Oracle2023.07 | 17 | 36.8 | |
| ReACTBackbone=Text-Davinci-003, Retriever=Oracle2023.07 | 8.5 | 29.8 | |
| ReACTBackbone=Claude-2, Retriever=Oracle2023.07 | 6 | 35 | |
| ReACT & DFSDTBackbone=Vicuna, Retriever=Oracle2023.07 | 0 | 0 | |
| ReACT & DFSDTBackbone=Alpaca, Retriever=Oracle2023.07 | 0 | 0 |