Multi-Turn Function Calling on BFCL Multi-Turn Base v3
69Greedy Success RateClaude-Sonnet-4.5
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude-Sonnet-4.5CoEvolve=false2026.04 | 69 | |
| Qwen3-30B-A3BCoEvolve=true2026.04 | 67 | |
| Qwen3-4B-InstructCoEvolve=true2026.04 | 63 | |
| Qwen2.5-7B-InstructCoEvolve=true2026.04 | 61.5 | |
| GPT-4CoEvolve=false2026.04 | 54 | |
| CuESThink=true, LLM=Qwen2.5, Params=14B2025.12 | 44.2 | |
| o3Think=true, LLM=OpenAI o3, Params=/2025.12 | 44 | |
| DeepSeek-V3Think=false, LLM=DeepSeek, Params=/2025.12 | 43.5 | |
| GPT-4o-miniThink=false, LLM=OpenAI, Params=/2025.12 | 43.5 | |
| Qwen3-30B-A3BCoEvolve=false2026.04 | 43.5 | |
| Amazon-Nova-Pro-v1:0Think=true, LLM=Amazon, Params=/2025.12 | 42.5 | |
| Gemini-2.5-FlashCoEvolve=false2026.04 | 41.5 | |
| DeepSeek-V3.2CoEvolve=false2026.04 | 41.5 | |
| Gemini-2.5-FlashThink=true, LLM=Gemini, Params=/2025.12 | 36 | |
| Gemini-2.5-ProThink=true, LLM=Gemini, Params=/2025.12 | 35 | |
| GPT-5Think=false, LLM=OpenAI, Params=/2025.12 | 33.5 | |
| GPT-5-nanoThink=false, LLM=OpenAI, Params=/2025.12 | 33.5 | |
| GPT-5-miniThink=false, LLM=OpenAI, Params=/2025.12 | 31.5 | |
| Amazon-Nova-Lite-v1:0Think=false, LLM=Amazon, Params=/2025.12 | 29 | |
| Qwen3-4B-InstructCoEvolve=false2026.04 | 26.5 | |
| LLaMA-3.3-70BCoEvolve=false2026.04 | 26 | |
| Gemma-3-27BCoEvolve=false2026.04 | 16.5 | |
| GPT-OSS-20BCoEvolve=false2026.04 | 14 | |
| Qwen2.5-7B-InstructCoEvolve=false2026.04 | 13.5 |