Tool execution on ToolBench With Retriever (End-to-End)
59.2SoPR (I1)ToolOmni
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ToolOmniInteraction budget (turns)=62026.04 | 59.2 | 58.1 | 45.1 | 54.13 | 50.31 | 57.55 | 42.62 | 50.16 | |
| GPT-3.5Interaction budget (turns)=62026.04 | 51.43 | 41.19 | 34.43 | 42.35 | 43.56 | 46.23 | 29.51 | 39.77 | |
| ToolLlama-v2Interaction budget (turns)=62026.04 | 43.1 | 46.7 | 40.2 | 43.33 | 42.94 | 50 | 14.75 | 35.9 | |
| ToolGenModel size (parameters)=7B, Interaction budget (turns)=62026.04 | 39.6 | 40.6 | 23 | 34.4 | 30.06 | 29.25 | 16.39 | 25.23 | |
| ToolGenModel size (parameters)=3B, Interaction budget (turns)=62026.04 | 37.4 | 40.6 | 30.3 | 36.1 | 36.2 | 29.25 | 9.84 | 25.1 | |
| ToolLlama-v1Interaction budget (turns)=62026.04 | 28.05 | 29.2 | 13.9 | 23.72 | 26.38 | 33.96 | 11.48 | 23.94 |