Tool-calling on BFCL (Multi-turn/Single-turn Evaluation)
63.1Multi-turn Success RateGemini-3-Pro-Preview
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini-3-Pro-PreviewAccess=Closed-source, Evaluation Protocol=Function Calling (FC)2026.05 | 63.1 | 83.8 | 73.4 | |
| Claude Sonnet 4.5Access=Closed-source, Evaluation Protocol=Function Calling (FC)2026.05 | 61.4 | 84.9 | 73.2 | |
| Qwen3-235B-A22B-Instruct-2507Access=Open-source, Evaluation Protocol=Function Calling (FC)2026.05 | 45.4 | 53.2 | 49.3 | |
| Qwen3-4B-RLAccess=Open-source, Training Mode=Reinforcement Learning (RL)2026.05 | 39.4 | 84.8 | 62.1 | |
| GPT-4.1-2025-04-14Access=Closed-source, Evaluation Protocol=Function Calling (FC)2026.05 | 38.9 | 76.4 | 57.7 | |
| Qwen3-4B w. strongerAccess=Open-source, Training Mode=SFT on stronger2026.05 | 37.2 | 84.8 | 61 | |
| Grok-4-0709Access=Closed-source, Evaluation Protocol=Function Calling (FC)2026.05 | 33.9 | 80.5 | 57.2 | |
| Qwen3-4B w. bfclAccess=Open-source, Training Mode=SFT on bfcl2026.05 | 22.7 | 83.9 | 53.3 | |
| Mistral-Large-2411Access=Open-source, Evaluation Protocol=Function Calling (FC)2026.05 | 14.1 | 83.3 | 48.7 | |
| Llama-4-Scout-17B-16E-InstructAccess=Open-source, Evaluation Protocol=Function Calling (FC)2026.05 | 9 | 82.1 | 45.5 | |
| Nova-2-Lite-v1:0Access=Closed-source, Evaluation Protocol=Function Calling (FC)2026.05 | 2.1 | 83.9 | 43 |