Function Calling on BFCL v4
68.8ScoreClaude-Sonnet-4.5
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Claude-Sonnet-4.5Model Type=Proprietary2025.12 | 68.8 | — | — | |
| Gemma-4-31B-it-NVFP4Backbone=Gemma-4-31B-it, Quantization Method=NVFP4 W4A42026.05 | 68.69 | — | — | |
| Gemma-4-31B-itBackbone=Gemma-4-31B-it, Quantization Method=BF162026.05 | 68.6 | — | — | |
| Mix-QuantBackbone=Gemma-4-31B-it, Quantization Method=Mix-Quant2026.05 | 68.19 | — | — | |
| DeepSeek-V3.1-Nex-N1Model Type=Open Source, Parameter Scale=> 100B, Function Call (FC) format=true, Search API=Google Search2025.12 | 65.3 | — | — | |
| GLM-4.6Model Type=Open Source, Parameter Scale=> 100B2025.12 | 62.1 | — | — | |
| GPT-5Model Type=Proprietary2025.12 | 61.6 | — | — | |
| Qwen3-32B-Nex-N1Model Type=Open Source, Parameter Scale=< 100B, Function Call (FC) format=true, Search API=Google Search2025.12 | 60.5 | — | — | |
| Kimi-K2-thinkingModel Type=Open Source, Parameter Scale=> 100B2025.12 | 60.4 | — | — | |
| Qwen3.5-9BBackbone=Qwen3.5-9B, Quantization Method=BF162026.05 | 58.96 | — | — | |
| Mix-QuantBackbone=Qwen3.5-9B, Quantization Method=Mix-Quant2026.05 | 57.89 | — | — | |
| Qwen3.5-9B-NVFP4Backbone=Qwen3.5-9B, Quantization Method=NVFP4 W4A42026.05 | 56.86 | — | — | |
| DeepSeek-V3.1Model Type=Open Source, Parameter Scale=> 100B2025.12 | 56.3 | — | — | |
| Gemini-2.5-proModel Type=Proprietary2025.12 | 54 | — | — | |
| Gemma-4-26B-A4B-itBackbone=Gemma-4-26B-A4B-it, Quantization Method=BF162026.05 | 53.07 | — | — | |
| Mix-QuantBackbone=Gemma-4-26B-A4B-it, Quantization Method=Mix-Quant2026.05 | 51.94 | — | — | |
| Qwen3-30B-A3B-Nex-N1Model Type=Open Source, Parameter Scale=< 100B, Function Call (FC) format=true, Search API=Google Search2025.12 | 51.9 | — | — | |
| Gemma-4-26B-A4B-it-NVFP4Backbone=Gemma-4-26B-A4B-it, Quantization Method=NVFP4 W4A42026.05 | 48.13 | — | — | |
| Qwen3-32BModel Type=Open Source, Parameter Scale=< 100B2025.12 | 45.4 | — | — | |
| InternLM3-8B-Nex-N1Model Type=Open Source, Parameter Scale=< 100B, Function Call (FC) format=true, Search API=Google Search2025.12 | 44.5 | — | — | |
| Mix-QuantBackbone=Qwen3-8B, Quantization Method=Mix-Quant2026.05 | 40.63 | — | — | |
| Qwen3-8BBackbone=Qwen3-8B, Quantization Method=BF162026.05 | 40.5 | — | — | |
| Qwen3-8B-NVFP4Backbone=Qwen3-8B, Quantization Method=NVFP4 W4A42026.05 | 38.77 | — | — | |
| Qwen3-30B-A3BModel Type=Open Source, Parameter Scale=< 100B2025.12 | 37.5 | — | — | |
| Minimax-M2Model Type=Open Source, Parameter Scale=> 100B2025.12 | 34.2 | — | — | |
| GRPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | — | 46.25 | 55 | |
| GRPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | — | 44.5 | 53 | |
| GSPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | — | 50.75 | 56 | |
| GSPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | — | 45.75 | 55 | |
| PPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | — | 44.75 | 56 | |
| PPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | — | 43.75 | 52 | |
| Qwen2.5-14BBackbone=Qwen2.5-14B, Fine-tuning=None2026.02 | — | 25.5 | 35.75 | |
| Qwen3-14BBackbone=Qwen3-14B, Fine-tuning=None2026.02 | — | 40.25 | 47 | |
| SeeUPOBackbone=Qwen2.5-14B, Fine-tuning=RLHF2026.02 | — | 55.25 | 57 | |
| SeeUPOBackbone=Qwen3-14B, Fine-tuning=RLHF2026.02 | — | 58 | 65 |