Function Calling on BFCL Simple Python
0.938AccuracyFull-FT
Evaluation results
| Method | Links | |
|---|---|---|
| Full-FTBackbone=Qwen3-14B-Base, Decode Execution=Independent2026.03 | 0.938 | |
| SUNBackbone=Qwen3-14B-Base, Decode Execution=Shared2026.03 | 0.935 | |
| Full-FTBackbone=Qwen3-8B-Base, Decode Execution=Independent2026.03 | 0.933 | |
| SUNBackbone=Qwen3-8B-Base, Decode Execution=Shared2026.03 | 0.933 | |
| PRISMBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Proposers=3, Reviewer=1, Synthesis iterations=32026.02 | 0.923 | |
| Full-FTBackbone=LLaMA3.1-8B, Decode Execution=Independent2026.03 | 0.9 | |
| SUNBackbone=LLaMA3.1-8B, Decode Execution=Shared2026.03 | 0.898 | |
| Full-FTBackbone=Qwen3-1.7B-Base, Decode Execution=Independent2026.03 | 0.898 | |
| Qwen3-235B-A22BType=Single-model reference, Zero-shot prompting=true, Thinking mode=false2026.02 | 0.89 | |
| Two HeadsBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Solvers=K=32026.02 | 0.888 | |
| SUNBackbone=Qwen3-1.7B-Base, Decode Execution=Shared2026.03 | 0.888 | |
| Qwen3-14B-BaseDecode Execution=N/A2026.03 | 0.88 | |
| MoABase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Agents per layer=32026.02 | 0.858 | |
| DeepSeek-V3.2Type=Single-model reference, Zero-shot prompting=true, Thinking mode=false2026.02 | 0.835 | |
| Qwen3-1.7B-BaseDecode Execution=N/A2026.03 | 0.825 | |
| Self-ConsistencyBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Sampling=K=5 paths2026.02 | 0.823 | |
| ReConcileBase Model=Qwen3-30B-A3B, Zero-shot prompting=true, Thinking mode=false, Agents=3, Discussion rounds=22026.02 | 0.823 | |
| Qwen3-30B-A3BType=Single-model reference, Zero-shot prompting=true, Thinking mode=false2026.02 | 0.818 | |
| Qwen3-8B-BaseDecode Execution=N/A2026.03 | 0.815 | |
| LLaMA3.1-8BDecode Execution=N/A2026.03 | 0.705 |