Function Calling on Berkeley Function Call Leaderboard (BFCL) Live (Out-of-Domain)
0.876AST SimpleQwen3-4B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Qwen3-4BParameters=4B2026.02 | 0.876 | 0.8 | 0.75 | 0.833 | 0.833 | |
| Qwen3-8BParameters=8B2026.02 | 0.849 | 0.794 | 0.625 | 0.75 | 0.834 | |
| GPT-4-turboModel Type=Private2026.02 | 0.837 | 0.786 | 0.812 | 0.708 | 0.81 | |
| GPT-4oModel Type=Private2026.02 | 0.814 | 0.788 | 0.875 | 0.75 | 0.826 | |
| Qwen2.5-32B-InstructParameters=32B2026.02 | 0.802 | 0.801 | 0.438 | 0.625 | 0.822 | |
| GPT-4o-miniModel Type=Private2026.02 | 0.787 | 0.762 | 0.875 | 0.708 | 0.799 | |
| BigAgent-8BParameters=8B2026.02 | 0.779 | 0.774 | 0.875 | 0.708 | 0.816 | |
| Qwen2.5-14B-InstructParameters=14B2026.02 | 0.771 | 0.75 | 0.75 | 0.708 | 0.79 | |
| Hammer2.1-7BParameters=7B2026.02 | 0.767 | 0.774 | 0.812 | 0.708 | 0.816 | |
| Qwen3-1.7BParameters=1.7B2026.02 | 0.756 | 0.727 | 0.75 | 0.75 | 0.769 | |
| Qwen2.5-7B-InstructParameters=7B2026.02 | 0.756 | 0.756 | 0.688 | 0.667 | 0.796 | |
| HyFunc♣LML=ToolACE-8B, LMS=Qwen3-0.6B2026.02 | 0.752 | 0.755 | 0.75 | 0.708 | 0.801 | |
| xLAM-8x7B-rParameters=8x7B2026.02 | 0.748 | 0.793 | 0.438 | 0.583 | 0.743 | |
| HyFunc♥LML=ToolACE-8B, LMS=Qwen2.5-0.5B-Instruct2026.02 | 0.748 | 0.752 | 0.688 | 0.667 | 0.786 | |
| Qwen2.5-3B-InstructParameters=3B2026.02 | 0.74 | 0.721 | 0.625 | 0.458 | 0.749 | |
| Qwen2.5-1.5B-InstructParameters=1.5B2026.02 | 0.74 | 0.661 | 0.5 | 0.458 | 0.727 | |
| ToolACE-8BParameters=8B2026.02 | 0.733 | 0.767 | 0.812 | 0.708 | 0.81 | |
| Hammer2.1-3BParameters=3B2026.02 | 0.733 | 0.733 | 0.625 | 0.667 | 0.79 | |
| GLM4-9B-chatParameters=9B2026.02 | 0.725 | 0.644 | 0 | 0 | 0.551 | |
| xLAM-7B-rParameters=7B2026.02 | 0.721 | 0.749 | 0.5 | 0.625 | 0.766 | |
| Hammer2.1-1.5BParameters=1.5B2026.02 | 0.713 | 0.698 | 0.5 | 0.625 | 0.755 | |
| Granite-20BParameters=20B2026.02 | 0.682 | 0.563 | 0.438 | 0.583 | 0.786 | |
| Qwen3-0.6BParameters=0.6B2026.02 | 0.659 | 0.544 | 0.375 | 0.542 | 0.622 | |
| xLAM-1B-rParameters=1B2026.02 | 0.64 | 0.534 | 0.062 | 0 | 0.512 | |
| Hammer2.1-0.5BParameters=0.5B2026.02 | 0.601 | 0.58 | 0.5 | 0.458 | 0.635 | |
| Qwen2.5-0.5B-InstructParameters=0.5B2026.02 | 0.562 | 0.413 | 0.562 | 0.208 | 0.524 |