Tool Planning on ToolBench G1 set
88.192Win Rate (G1-Instruction)GPT4 TOPGUN
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT4 TOPGUNReference model=ChatGPT REACT, Evaluation setting=Gray Box2024.02 | 88.192 | 87.46 | 87.15 | |
| GPT4 TOPGUNReference model=T.LLAMA ReACT, Evaluation setting=Gray Box2024.02 | 86.72 | 82.94 | 80.8 | |
| GPT4 TOPGUNReference model=GPT4 ReACT, Evaluation setting=Gray Box2024.02 | 82.996 | 79.956 | 77.633 | |
| GPT4 TOPGUNReference model=GPT4 DFSDT, Evaluation setting=Gray Box2024.02 | 82.065 | 73.69 | 71.14 | |
| GPT4 TOPGUNReference model=T.LLAMA DFSDT, Evaluation setting=Gray Box2024.02 | 81.75 | 75.51 | 73.81 | |
| GPT4 TOPGUNReference model=T.LLAMA DFSDT+Ret, Evaluation setting=Gray Box2024.02 | 80.35 | 77.11 | 75.39 | |
| GPT4 TOPGUNReference model=ChatGPT DFSDT, Evaluation setting=Gray Box2024.02 | 78.49 | 77.55 | 76.24 | |
| GPT4 DFSDTReference model=ChatGPT REACT, Evaluation setting=Gray Box2024.02 | 67.5 | 67.8 | 66.5 | |
| T.LLAMA DFSDT+RetReference model=ChatGPT REACT, Evaluation setting=Gray Box2024.02 | 62.3 | 59 | 55 | |
| ChatGPT DFSDTReference model=ChatGPT REACT, Evaluation setting=Gray Box2024.02 | 60.5 | 62 | 57.3 | |
| GPT4 ReACTReference model=ChatGPT REACT, Evaluation setting=Gray Box2024.02 | 60 | 58.8 | 63.5 | |
| T.LLAMA DFSDTReference model=ChatGPT REACT, Evaluation setting=Gray Box2024.02 | 55 | 55.3 | 54.5 | |
| T.LLAMA REACTReference model=ChatGPT REACT, Evaluation setting=Gray Box2024.02 | 45 | 42 | 47.5 |