Tool-use Inference on API-Bank
5.8Match Rate (#MAT)TOOLSPEC
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TOOLSPECModel=LLaMA-3.1-8B-Instruct2026.04 | 5.8 | 125.59 | 4.45 | 4.19 | |
| TOOLSPECModel=LLaMA-3.2-3B-Instruct2026.04 | 5.2 | 133.3 | 4.09 | 3.66 | |
| TOOLSPECModel=Qwen2.5-7B-Instruct2026.04 | 4.9 | 129.4 | 3.95 | 3.67 | |
| TOOLSPECModel=Qwen2.5-14B-Instruct2026.04 | 4.43 | 75.27 | 3.86 | 3.48 | |
| Eagle-2*Model=Qwen2.5-14B-Instruct2026.04 | 3.87 | 46.8 | 2.4 | 2.12 | |
| SAMDModel=Qwen2.5-7B-Instruct2026.04 | 3.6 | 83.96 | 2.56 | 2.22 | |
| TRModel=LLaMA-3.1-8B-Instruct2026.04 | 3.34 | 71.1 | 2.52 | 2.45 | |
| Eagle-3*Model=LLaMA-3.1-8B-Instruct2026.04 | 3.34 | 47.24 | 1.67 | 1.56 | |
| SAMDModel=LLaMA-3.1-8B-Instruct2026.04 | 3.2 | 69.68 | 2.47 | 2.41 | |
| TRModel=LLaMA-3.2-3B-Instruct2026.04 | 3.11 | 75.5 | 2.32 | 2.32 | |
| TRModel=Qwen2.5-7B-Instruct2026.04 | 3.09 | 78.95 | 2.41 | 2.28 | |
| TRModel=Qwen2.5-14B-Instruct2026.04 | 3.01 | 46.94 | 2.41 | 2.34 | |
| SAMDModel=LLaMA-3.2-3B-Instruct2026.04 | 2.99 | 72.79 | 2.18 | 2.17 | |
| SAMDModel=Qwen2.5-14B-Instruct2026.04 | 2.93 | 45.86 | 2.35 | 2.19 | |
| PLDModel=Qwen2.5-7B-Instruct2026.04 | 2.37 | 64.56 | 1.97 | 1.91 | |
| PLDModel=LLaMA-3.1-8B-Instruct2026.04 | 2.35 | 60.72 | 2.15 | 2.12 | |
| PLDModel=LLaMA-3.2-3B-Instruct2026.04 | 2.19 | 69.81 | 2.14 | 1.99 | |
| PLDModel=Qwen2.5-14B-Instruct2026.04 | 1.98 | 35.3 | 1.81 | 1.82 | |
| VanillaModel=Qwen2.5-7B-Instruct2026.04 | 1 | 32.76 | 1 | 1 | |
| VanillaModel=Qwen2.5-14B-Instruct2026.04 | 1 | 19.5 | 1 | 1 | |
| VanillaModel=LLaMA-3.1-8B-Instruct2026.04 | 1 | 28.21 | 1 | 1 | |
| VanillaModel=LLaMA-3.2-3B-Instruct2026.04 | 1 | 32.6 | 1 | 1 |