Tool Execution on ToolBench With Ground-Truth Tools
60.7SoPR (I1)ToolOmni
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ToolOmniInteraction budget (turns)=62026.04 | 60.7 | 50.9 | 52.5 | 54.7 | 52.76 | 56.6 | 40.98 | 50.11 | |
| GPT-3.5Interaction budget (turns)=62026.04 | 56.6 | 47.8 | 54.64 | 53.01 | — | — | — | — | |
| ToolGenModel size (parameters)=7B, Interaction budget (turns)=62026.04 | 44.2 | 42.9 | 32 | 39.7 | 35.58 | 45.28 | 16.39 | 32.42 | |
| ToolLlama-v2Interaction budget (turns)=62026.04 | 43.6 | 45.8 | 39.3 | 42.9 | 38.65 | 55.66 | 32.79 | 42.37 | |
| ToolGenModel size (parameters)=3B, Interaction budget (turns)=62026.04 | 40.8 | 43.4 | 21.3 | 35.17 | 39.26 | 47.17 | 14.75 | 33.73 | |
| ToolLlama-v1Interaction budget (turns)=62026.04 | 24.2 | 29.2 | 12.3 | 21.9 | 28.22 | 42.45 | 16.39 | 29.02 |