Function Calling on BFCL Live and Non-live
90.8Non-live AST ScoreQwen2.5-72B-it
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Qwen2.5-72B-itModel size=72B2025.12 | 90.8 | 92.7 | 75.3 | 100 | 89.7 | |
| GPT42025.12 | 88.1 | 89.4 | 79.8 | 83.3 | 85.2 | |
| ToolACE-8BModel size=8B2025.12 | 87.5 | 89.2 | 78.5 | 83.3 | 84.6 | |
| MoRAgent-PhiBase Model=Phi3.5-mini-Instruct, Number of Parameters=3.82B+0.36B2025.12 | 83 | 78.2 | 72.7 | 94.4 | 82.1 | |
| MiniCPM3-4BModel size=4B2025.12 | 80.8 | 87.6 | 70 | 72.2 | 77.7 | |
| Llama3.2-3B-itModel size=3B2025.12 | 80.6 | 83.7 | 55.8 | 88.9 | 77.3 | |
| MoRAgent-openPanguBase Model=openPangu-Embedded-1B, Number of Parameters=1.16B+0.20B2025.12 | 76.9 | 72.9 | 64.7 | 94.4 | 77.2 | |
| MoRAgent-LlamaBase Model=Llama3.2-1B-Instruct, Number of Parameters=1.24B+0.16B2025.12 | 75.2 | 80 | 60.7 | 94.4 | 77.6 | |
| BaseBase Model=Phi3.5-mini-Instruct, Number of Parameters=3.82B2025.12 | 69 | 57.3 | 58.2 | 72.2 | 64.2 | |
| BaseBase Model=openPangu-Embedded-1B, Number of Parameters=1.16B2025.12 | 23 | 23.9 | 31.5 | 16.7 | 23.8 | |
| BaseBase Model=Llama3.2-1B-Instruct, Number of Parameters=1.24B2025.12 | 21.9 | 19.2 | 29.8 | 38.9 | 27.5 |