Function Calling on BFCL Multi-Turn v3
78.7Overall AccuracyMIND-Skill
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| MIND-SkillSkill Generation=GPT-5.4, Inference Model=Qwen3.5-122B-A10B2026.05 | 78.7 | — | — | — | — | |
| MIND-SkillSkill Generation=Qwen3.5-122B-A10B, Inference Model=Qwen3.5-122B-A10B2026.05 | 77.3 | — | — | — | — | |
| ACESkill Generation=Qwen3.5-122B-A10B, Inference Model=Qwen3.5-122B-A10B2026.05 | 74 | — | — | — | — | |
| Trace2SkillSkill Generation=Qwen3.5-122B-A10B, Inference Model=Qwen3.5-122B-A10B2026.05 | 72.7 | — | — | — | — | |
| ACESkill Generation=GPT-5.4, Inference Model=Qwen3.5-122B-A10B2026.05 | 72.7 | — | — | — | — | |
| Trace2SkillSkill Generation=GPT-5.4, Inference Model=Qwen3.5-122B-A10B2026.05 | 71.3 | — | — | — | — | |
| Skill-extractSkill Generation=GPT-5.4, Inference Model=Qwen3.5-122B-A10B2026.05 | 69.3 | — | — | — | — | |
| Skill-extractSkill Generation=Qwen3.5-122B-A10B, Inference Model=Qwen3.5-122B-A10B2026.05 | 68.7 | — | — | — | — | |
| Claude-Opus-4.5Temperature=0.62026.01 | 68.38 | 81 | 64 | 58 | 70.5 | |
| GLM-4.6Temperature=0.62026.01 | 68 | 74.5 | 68 | 63 | 66.5 | |
| ICLSkill Generation=None, Inference Model=Qwen3.5-122B-A10B2026.05 | 64.7 | — | — | — | — | |
| Astra-32B-thinking-v1Temperature=0.62026.01 | 64.25 | 76.5 | 65.5 | 48.5 | 66.5 | |
| ReActSkill Generation=None, Inference Model=Qwen3.5-122B-A10B2026.05 | 63.3 | — | — | — | — | |
| Gemini-3-ProTemperature=0.62026.01 | 63.13 | 69 | 63 | 56.5 | 64 | |
| Claude-Sonnet-4.5Temperature=0.62026.01 | 61.38 | 69 | 65 | 52.5 | 59 | |
| Astra-14B-thinking-v1Temperature=0.62026.01 | 58.13 | 67 | 56 | 48.5 | 61 | |
| LoopTool-32BTemperature=0.62026.01 | 57.75 | 66.5 | 58 | 44.5 | 62 | |
| Claude-Haiku-4.5Temperature=0.62026.01 | 53.63 | 63.5 | 42.5 | 52.5 | 56 | |
| Kimi-K2-InstructTemperature=0.62026.01 | 50.63 | 62 | 41 | 44.5 | 55 | |
| Qwen3-32BTemperature=0.62026.01 | 49.63 | 59 | 47.5 | 40.5 | 51.5 | |
| Kimi-K2-Instruct-09052026.01 | 45.88 | 57.5 | 35 | 42 | 49 | |
| Qwen3-235B-Thinking-2507Mode=Thinking2026.01 | 45.75 | 60 | 35 | 34 | 54 | |
| Qwen3-14BTemperature=0.62026.01 | 44.5 | 54 | 39.5 | 39 | 45.5 | |
| Qwen3-235B-Instruct-2507Mode=Instruct2026.01 | 42.25 | 58 | 33 | 27.5 | 50.5 | |
| EnvScalerBackbone=Qwen3-8B, Training=SFT & RL2026.01 | 41.88 | 55.5 | 36 | 35 | 41 | |
| GPT-4.12026.01 | 40 | 46 | 37.5 | 32 | 44.5 | |
| Llama-3.1-70B + ToolWeave (G)Model Family=Llama-3.1-70B, Fine-tuning Data=ToolWeave (G)2026.04 | 39.75 | 42.5 | 43 | 37 | 36.5 | |
| GPT-4.1Temperature=0.62026.01 | 38.88 | 47.5 | 32.5 | 32.5 | 43 | |
| EnvScalerBackbone=Qwen3-4B, Training=SFT & RL2026.01 | 38 | 51 | 34 | 28 | 39 | |
| Llama-3.1-70B + ToolWeave-R (G)Model Family=Llama-3.1-70B, Fine-tuning Data=ToolWeave-R (G)2026.04 | 37.38 | 40 | 43 | 34 | 32.5 | |
| EnvScalerBackbone=Qwen3-8B, Training=SFT2026.01 | 37 | 47 | 33 | 29.5 | 38.5 | |
| EnvScalerBackbone=Qwen3-4B, Training=SFT2026.01 | 34.88 | 47 | 24 | 31.5 | 37 | |
| Llama-3.1-70B + ToolWeave (O)Model Family=Llama-3.1-70B, Fine-tuning Data=ToolWeave (O)2026.04 | 33.25 | 37.5 | 35 | 32 | 28.5 | |
| Qwen3-32B + ARTIS (Sequential)Model backbone=Qwen3-32B, Test-time scaling method=ARTIS (Sequential)2026.02 | 30.5 | 41 | 34.5 | 24 | 22.5 | |
| Gemini-2.5-ProTemperature=0.62026.01 | 30.12 | 28.5 | 35 | 30 | 27 | |
| Phi-4 + ToolWeave (G)Model Family=Phi-4, Fine-tuning Data=ToolWeave (G)2026.04 | 29.5 | 36 | 30.5 | 31.5 | 20 | |
| Qwen3-8BMode=Thinking2026.01 | 28.88 | 32 | 33.5 | 22 | 28 | |
| Qwen3-32B + ARTIS (Parallel)Model backbone=Qwen3-32B, Test-time scaling method=ARTIS (Parallel)2026.02 | 28.75 | 41 | 27.5 | 21 | 25.5 | |
| Qwen3-32B + Sequential RevisionModel backbone=Qwen3-32B, Test-time scaling method=Sequential Revision2026.02 | 27 | 35.5 | 27 | 22 | 23.5 | |
| Phi-4 + ToolWeave-R (G)Model Family=Phi-4, Fine-tuning Data=ToolWeave-R (G)2026.04 | 26.25 | 33.5 | 33.5 | 20.5 | 17.5 | |
| Qwen3-14B + ARTIS (Sequential)Model backbone=Qwen3-14B, Test-time scaling method=ARTIS (Sequential)2026.02 | 25.75 | 36.5 | 26.5 | 20.5 | 19.5 | |
| Qwen3-4BMode=Thinking2026.01 | 25.38 | 32 | 20 | 24 | 25.5 | |
| Phi-4 + ToolWeave (O)Model Family=Phi-4, Fine-tuning Data=ToolWeave (O)2026.04 | 24.5 | 24 | 28.5 | 31 | 14.5 | |
| Qwen3-14B + Sequential RevisionModel backbone=Qwen3-14B, Test-time scaling method=Sequential Revision2026.02 | 24.25 | 36.5 | 20 | 19 | 21.5 | |
| Qwen3-32B + Weighted BoNModel backbone=Qwen3-32B, Test-time scaling method=Weighted BoN2026.02 | 24 | 36 | 23 | 20 | 17 | |
| Llama-3.1-70B + ToolFlowModel Family=Llama-3.1-70B, Fine-tuning Data=ToolFlow2026.04 | 23.5 | 29.5 | 20.5 | 20 | 24 | |
| EnvScalerBackbone=Qwen3-1.7B, Training=SFT & RL2026.01 | 23 | 31.5 | 17 | 20.5 | 23 | |
| Qwen3-14B + ARTIS (Parallel)Model backbone=Qwen3-14B, Test-time scaling method=ARTIS (Parallel)2026.02 | 22.5 | 36 | 20 | 17 | 17 | |
| Qwen3-32BModel backbone=Qwen3-32B, Test-time scaling method=None (Direct execution)2026.02 | 21.75 | 30.5 | 23.5 | 17 | 16 | |
| Llama-3.1-8B + ToolWeave (G)Model Family=Llama-3.1-8B, Fine-tuning Data=ToolWeave (G)2026.04 | 21.12 | 28 | 18 | 18.5 | 20 | |
| Qwen3-14B + Weighted BoNModel backbone=Qwen3-14B, Test-time scaling method=Weighted BoN2026.02 | 21 | 34 | 20.5 | 12 | 17.5 | |
| Llama-3.1-8B + ToolWeave (O)Model Family=Llama-3.1-8B, Fine-tuning Data=ToolWeave (O)2026.04 | 19.88 | 23 | 21.5 | 15.5 | 19.5 | |
| Qwen3-8B + ARTIS (Sequential)Model backbone=Qwen3-8B, Test-time scaling method=ARTIS (Sequential)2026.02 | 18.75 | 29.5 | 16.5 | 12.5 | 16.5 | |
| Qwen3-14BModel backbone=Qwen3-14B, Test-time scaling method=None (Direct execution)2026.02 | 18.75 | 29 | 18 | 12 | 16 | |
| Llama-3.1-8B + ToolWeave-R (G)Model Family=Llama-3.1-8B, Fine-tuning Data=ToolWeave-R (G)2026.04 | 18.38 | 24 | 18 | 14.5 | 17 | |
| EnvScalerBackbone=Qwen3-1.7B, Training=SFT2026.01 | 18.13 | 24.5 | 11.5 | 20 | 16.5 | |
| Qwen3-8B + ARTIS (Parallel)Model backbone=Qwen3-8B, Test-time scaling method=ARTIS (Parallel)2026.02 | 17.75 | 27.5 | 11.5 | 16 | 16 | |
| Qwen3-8B + Sequential RevisionModel backbone=Qwen3-8B, Test-time scaling method=Sequential Revision2026.02 | 16.5 | 25.5 | 16.5 | 12.5 | 11.5 | |
| Qwen3-8B + Weighted BoNModel backbone=Qwen3-8B, Test-time scaling method=Weighted BoN2026.02 | 15.5 | 24 | 16 | 10.5 | 11.5 | |
| Qwen3-8BModel backbone=Qwen3-8B, Test-time scaling method=None (Direct execution)2026.02 | 12.88 | 21 | 11.5 | 9.5 | 9.5 | |
| Llama-3.1-70B (Base)Model Family=Llama-3.1-70B, Fine-tuning Data=None2026.04 | 12.5 | 17 | 13 | 10.5 | 9.5 | |
| Phi-4 + ToolFlowModel Family=Phi-4, Fine-tuning Data=ToolFlow2026.04 | 10.12 | 11 | 10 | 11.5 | 8 | |
| Qwen3-1.7BMode=Thinking2026.01 | 9.75 | 13.5 | 6 | 12.5 | 7 | |
| Llama-3.1-8B (Base)Model Family=Llama-3.1-8B, Fine-tuning Data=None2026.04 | 9.25 | 11 | 8 | 8.5 | 9.5 | |
| Llama-3.1-8B + ToolFlowModel Family=Llama-3.1-8B, Fine-tuning Data=ToolFlow2026.04 | 7.5 | 9.5 | 5.5 | 5.5 | 9.5 | |
| Llama-3.1-70B + ToolDialModel Family=Llama-3.1-70B, Fine-tuning Data=ToolDial2026.04 | 3.75 | 4.5 | 3 | 4 | 3.5 | |
| Phi-4 (Base)Model Family=Phi-4, Fine-tuning Data=None2026.04 | 3.12 | 7.5 | 0 | 2.5 | 2.5 | |
| Phi-4 + ToolDialModel Family=Phi-4, Fine-tuning Data=ToolDial2026.04 | 2 | 2 | 0.5 | 3.5 | 2 | |
| Llama-3.1-8B + ToolDialModel Family=Llama-3.1-8B, Fine-tuning Data=ToolDial2026.04 | 1.75 | 0.5 | 3.5 | 2.5 | 0.5 |