Tool-use Inference on BFCL v2
5.31MAT ScoreTOOLSPEC
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TOOLSPECModel=LLaMA-3.1-8B-Instruct2026.04 | 5.31 | 113.55 | 4.08 | 4.19 | |
| TOOLSPECModel=LLaMA-3.2-3B-Instruct2026.04 | 4.78 | 115.84 | 3.52 | 3.66 | |
| TOOLSPECModel=Qwen2.5-7B-Instruct2026.04 | 4.73 | 117.17 | 3.58 | 3.67 | |
| TOOLSPECModel=Qwen2.5-14B-Instruct2026.04 | 4.15 | 65.07 | 3.37 | 3.48 | |
| Eagle-2*Model=Qwen2.5-14B-Instruct2026.04 | 3.67 | 39.39 | 2.04 | 2.12 | |
| TRModel=LLaMA-3.1-8B-Instruct2026.04 | 3.29 | 66.79 | 2.4 | 2.45 | |
| Eagle-3*Model=LLaMA-3.1-8B-Instruct2026.04 | 3.17 | 42.86 | 1.54 | 1.56 | |
| SAMDModel=LLaMA-3.1-8B-Instruct2026.04 | 3.14 | 66.24 | 2.38 | 2.41 | |
| TRModel=LLaMA-3.2-3B-Instruct2026.04 | 3.06 | 76.35 | 2.32 | 2.32 | |
| TRModel=Qwen2.5-14B-Instruct2026.04 | 2.99 | 46.34 | 2.4 | 2.34 | |
| SAMDModel=Qwen2.5-7B-Instruct2026.04 | 2.92 | 70.7 | 2.16 | 2.22 | |
| TRModel=Qwen2.5-7B-Instruct2026.04 | 2.89 | 73.64 | 2.25 | 2.28 | |
| SAMDModel=Qwen2.5-14B-Instruct2026.04 | 2.78 | 42.29 | 2.19 | 2.19 | |
| SAMDModel=LLaMA-3.2-3B-Instruct2026.04 | 2.75 | 71.09 | 2.16 | 2.17 | |
| PLDModel=LLaMA-3.1-8B-Instruct2026.04 | 2.3 | 58.72 | 2.11 | 2.12 | |
| PLDModel=Qwen2.5-7B-Instruct2026.04 | 2.24 | 62.52 | 1.91 | 1.91 | |
| PLDModel=LLaMA-3.2-3B-Instruct2026.04 | 2.11 | 64.83 | 1.97 | 1.99 | |
| PLDModel=Qwen2.5-14B-Instruct2026.04 | 2.02 | 35.34 | 1.83 | 1.82 | |
| VanillaModel=Qwen2.5-7B-Instruct2026.04 | 1 | 32.73 | 1 | 1 | |
| VanillaModel=Qwen2.5-14B-Instruct2026.04 | 1 | 19.31 | 1 | 1 | |
| VanillaModel=LLaMA-3.1-8B-Instruct2026.04 | 1 | 27.83 | 1 | 1 | |
| VanillaModel=LLaMA-3.2-3B-Instruct2026.04 | 1 | 32.91 | 1 | 1 |