Tool Use on ToolBench 50 APIs v1 (test)
99.2WellformednessLlama-2-Chat-7B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Llama-2-Chat-7BSetting=Fine-tuning w/ STE2024.03 | 99.2 | 94.9 | 73.3 | |
| Mistral-Instruct-7BSetting=Fine-tuning w/ STE2024.03 | 99.1 | 95.8 | 76.8 | |
| Llama-2-Chat-13BSetting=Fine-tuning w/ STE2024.03 | 98.9 | 95.1 | 74.3 | |
| ToolLLaMA-v2Setting=Baseline2024.03 | 98.1 | 49 | 37.3 | |
| GPT-4 (8k-0613)Setting=ICL w/ STE2024.03 | 97.7 | 92.8 | 76.3 | |
| GPT-3.5-turbo (16k-0613)Setting=ICL w/ STE2024.03 | 97.6 | 90.8 | 75.6 | |
| GPT-3.5-turbo (16k-0613)Setting=Baseline2024.03 | 96.9 | 77.6 | 60.5 | |
| GPT-4 (8k-0613)Setting=Baseline2024.03 | 96.1 | 78.1 | 60.8 | |
| Llama-2-Chat-13BSetting=ICL w/ STE2024.03 | 87.5 | 86.6 | 62.9 | |
| Llama-2-Chat-13BSetting=Baseline2024.03 | 79.3 | 53.6 | 32.7 | |
| Mistral-Instruct-7BSetting=ICL w/ STE2024.03 | 69.9 | 88.4 | 47.9 | |
| Mistral-Instruct-7BSetting=Baseline2024.03 | 61.7 | 69.3 | 30.1 | |
| Llama-2-Chat-7BSetting=ICL w/ STE2024.03 | 58.3 | 86.7 | 41.7 | |
| Llama-2-Chat-7BSetting=Baseline2024.03 | 34.5 | 40.2 | 10.7 |