Execution Accuracy on BFCL v2
88.24Non-Live AST AccuracyQwen2.5-7B-Instruct-CAST
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen2.5-7B-Instruct-CASTCategory=Open Source2026.05 | 88.24 | 87.4 | 88.43 | |
| Qwen2.5-Coder-7B-Instruct-CASTCategory=Open Source2026.05 | 87.12 | 82.43 | 84.7 | |
| Qwen2.5-7B-Instruct-GRPOCategory=Open Source2026.05 | 87.05 | 80.29 | 83.67 | |
| GPT-4oCategory=Closed Source2026.05 | 86.83 | 78.92 | 82.88 | |
| Qwen2.5-Coder-7B-Instruct-GRPOCategory=Open Source2026.05 | 86.51 | 75.07 | 80.79 | |
| DeepSeek-V3Category=Closed Source2026.05 | 86.37 | 75.28 | 80.83 | |
| Qwen2.5-7B-Instruct-SFTCategory=Open Source2026.05 | 86.23 | 78.9 | 82.58 | |
| GraniteCategory=Other2026.05 | 86.17 | 79.19 | 84.71 | |
| Qwen2.5-Coder-7B-Instruct-SFTCategory=Open Source2026.05 | 86.07 | 74.92 | 80.86 | |
| GorillaCategory=Other2026.05 | 86.02 | 80.44 | 82.21 | |
| Qwen-MaxCategory=Closed Source2026.05 | 84.97 | 80.85 | 82.91 | |
| Llama-3.2-8B-Instruct-CASTCategory=Open Source2026.05 | 83.95 | 76.84 | 80.43 | |
| Llama-3.2-8B-Instruct-SFTCategory=Open Source2026.05 | 83.14 | 73.82 | 78.48 | |
| OTCCategory=Other2026.05 | 82.64 | 71.28 | 78.33 | |
| Llama-3.2-8B-Instruct-GRPOCategory=Open Source2026.05 | 82.18 | 75.21 | 78.73 | |
| ToolAlignCategory=Other2026.05 | 77.26 | 61.47 | 71.16 | |
| ToolformerCategory=Other2026.05 | 76.11 | 59.47 | 67.07 | |
| ReActCategory=Other2026.05 | 73.58 | 58.43 | 66.08 |