Tool usage simulation on ToolAlpaca evaluation
78.38Procedure ScoreToolCoder
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ToolCoderBackbone=gpt-4o-mini, Error Reflection=true, Reusability Mechanism=false2025.02 | 78.38 | 75.68 | 72.97 | |
| GPT-3.52023.12 | 77 | 85 | 75 | |
| ToolCoder (w/o Error Reflection)Backbone=gpt-4o-mini, Error Reflection=false, Reusability Mechanism=false2025.02 | 76.46 | 68.11 | 67.94 | |
| ToolAlpacaModel Size=13B, Attention Buckets=true2023.12 | 75 | 78 | 74 | |
| ToolAlpacaModel Size=13B2023.12 | 70 | 73 | 70 | |
| ToolAlpacaModel Size=7B, Attention Buckets=true2023.12 | 69 | 73 | 65 | |
| CodeActBackbone=gpt-4o-mini, Reusability Mechanism=false2025.02 | 68.92 | 58.11 | 56.94 | |
| ChameleonBackbone=gpt-4o-mini, Reusability Mechanism=false2025.02 | 68.06 | 58.33 | 57.92 | |
| ReActBackbone=gpt-4o-mini, Reusability Mechanism=false2025.02 | 64.86 | 60.81 | 54.05 | |
| ToolAlpacaModel Size=7B2023.12 | 63 | 69 | 60 | |
| VicunaModel Size=7B2023.12 | 19 | 21 | 17 | |
| VicunaModel Size=13B2023.12 | 17 | 31 | 16 |