Tool Usage on BFCL Parallel v1
95.5AccuracyTHINKBRAKE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| THINKBRAKEBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE2025.10 | 95.5 | -26.8 | |
| Qwen3-4B-ThinkingBase Model=Qwen3-4B-Thinking, Stopping Strategy=Base2025.10 | 93.5 | — | |
| THINKBRAKEBase Model=Qwen3-14B, Stopping Strategy=THINKBRAKE2025.10 | 93 | -21.2 | |
| Qwen3-32BBase Model=Qwen3-32B, Stopping Strategy=Base2025.10 | 93 | — | |
| Qwen3-14BBase Model=Qwen3-14B, Stopping Strategy=Base2025.10 | 92 | — | |
| NoWaitBase Model=Qwen3-32B, Stopping Strategy=NoWait2025.10 | 91 | -13 | |
| THINKBRAKEBase Model=Qwen3-32B, Stopping Strategy=THINKBRAKE2025.10 | 91 | -14.3 | |
| Qwen3-4BBase Model=Qwen3-4B, Stopping Strategy=Base2025.10 | 90 | — | |
| NoWaitBase Model=Qwen3-14B, Stopping Strategy=NoWait2025.10 | 90 | -16.3 | |
| THINKBRAKEBase Model=Qwen3-4B, Stopping Strategy=THINKBRAKE2025.10 | 89.5 | -16.5 | |
| NoWaitBase Model=Qwen3-4B-Thinking, Stopping Strategy=NoWait2025.10 | 88.5 | -44.4 | |
| NoWaitBase Model=Qwen3-4B, Stopping Strategy=NoWait2025.10 | 88 | -13.5 | |
| Phi-4-ReasoningBase Model=Phi-4-Reasoning, Stopping Strategy=Base2025.10 | 87.5 | — | |
| THINKBRAKEBase Model=Phi-4-Reasoning, Stopping Strategy=THINKBRAKE2025.10 | 84.5 | 2.7 | |
| NoWaitBase Model=Phi-4-Reasoning, Stopping Strategy=NoWait2025.10 | 81 | -1.3 | |
| ThinkLessBase Model=Qwen3-4B-Thinking, Stopping Strategy=ThinkLess2025.10 | 79.5 | -100 | |
| THINKBRAKE-pBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE-p2025.10 | 78 | -26.9 | |
| NoWaitBase Model=DeepSeek-R1-7B, Stopping Strategy=NoWait2025.10 | 62 | -4 | |
| DeepSeek-R1-7BBase Model=DeepSeek-R1-7B, Stopping Strategy=Base2025.10 | 52.5 | — | |
| THINKBRAKEBase Model=DeepSeek-R1-7B, Stopping Strategy=THINKBRAKE2025.10 | 42 | -14.6 |