Tool-use Inference on ToolAlpaca
5.26Match RateTOOLSPEC
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TOOLSPECModel=LLaMA-3.1-8B-Instruct2026.04 | 5.26 | 112.26 | 4.05 | 4.19 | |
| TOOLSPECModel=Qwen2.5-7B-Instruct2026.04 | 4.55 | 113.54 | 3.47 | 3.67 | |
| TOOLSPECModel=LLaMA-3.2-3B-Instruct2026.04 | 4.52 | 112.22 | 3.38 | 3.66 | |
| TOOLSPECModel=Qwen2.5-14B-Instruct2026.04 | 4.02 | 60.86 | 3.2 | 3.48 | |
| Eagle-2*Model=Qwen2.5-14B-Instruct2026.04 | 3.58 | 36.24 | 1.91 | 2.12 | |
| TRModel=LLaMA-3.1-8B-Instruct2026.04 | 3.32 | 67.69 | 2.44 | 2.45 | |
| SAMDModel=LLaMA-3.1-8B-Instruct2026.04 | 3.12 | 65.82 | 2.38 | 2.41 | |
| TRModel=LLaMA-3.2-3B-Instruct2026.04 | 3.03 | 73.45 | 2.31 | 2.32 | |
| Eagle-3*Model=LLaMA-3.1-8B-Instruct2026.04 | 2.93 | 40.37 | 1.46 | 1.56 | |
| TRModel=Qwen2.5-7B-Instruct2026.04 | 2.87 | 71.28 | 2.18 | 2.28 | |
| TRModel=Qwen2.5-14B-Instruct2026.04 | 2.85 | 41.97 | 2.21 | 2.34 | |
| SAMDModel=LLaMA-3.2-3B-Instruct2026.04 | 2.68 | 71.84 | 2.16 | 2.17 | |
| SAMDModel=Qwen2.5-7B-Instruct2026.04 | 2.62 | 63.57 | 1.94 | 2.22 | |
| SAMDModel=Qwen2.5-14B-Instruct2026.04 | 2.62 | 38.48 | 2.02 | 2.19 | |
| PLDModel=LLaMA-3.1-8B-Instruct2026.04 | 2.27 | 58.28 | 2.1 | 2.12 | |
| PLDModel=Qwen2.5-7B-Instruct2026.04 | 2.1 | 60.75 | 1.86 | 1.91 | |
| PLDModel=LLaMA-3.2-3B-Instruct2026.04 | 2.06 | 61.41 | 1.85 | 1.99 | |
| PLDModel=Qwen2.5-14B-Instruct2026.04 | 2 | 34.85 | 1.83 | 1.82 | |
| VanillaModel=Qwen2.5-7B-Instruct2026.04 | 1 | 32.72 | 1 | 1 | |
| VanillaModel=Qwen2.5-14B-Instruct2026.04 | 1 | 19.02 | 1 | 1 | |
| VanillaModel=LLaMA-3.1-8B-Instruct2026.04 | 1 | 27.69 | 1 | 1 | |
| VanillaModel=LLaMA-3.2-3B-Instruct2026.04 | 1 | 33.2 | 1 | 1 |