Tool Usage on BFCL Parallel v2
87.5AccuracyQwen3-4B-Thinking
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Qwen3-4B-ThinkingBase Model=Qwen3-4B-Thinking, Stopping Strategy=Base2025.10 | 87.5 | — | |
| THINKBRAKEBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE2025.10 | 87.5 | -32.3 | |
| NoWaitBase Model=Qwen3-4B-Thinking, Stopping Strategy=NoWait2025.10 | 81.3 | -50.7 | |
| Qwen3-4BBase Model=Qwen3-4B, Stopping Strategy=Base2025.10 | 81.2 | — | |
| Phi-4-ReasoningBase Model=Phi-4-Reasoning, Stopping Strategy=Base2025.10 | 81.2 | — | |
| NoWaitBase Model=Phi-4-Reasoning, Stopping Strategy=NoWait2025.10 | 81.2 | -6.4 | |
| NoWaitBase Model=Qwen3-4B, Stopping Strategy=NoWait2025.10 | 75 | -5.1 | |
| THINKBRAKEBase Model=Phi-4-Reasoning, Stopping Strategy=THINKBRAKE2025.10 | 75 | 4.6 | |
| Qwen3-32BBase Model=Qwen3-32B, Stopping Strategy=Base2025.10 | 68.8 | — | |
| NoWaitBase Model=Qwen3-32B, Stopping Strategy=NoWait2025.10 | 68.8 | -3.9 | |
| THINKBRAKEBase Model=Qwen3-32B, Stopping Strategy=THINKBRAKE2025.10 | 68.8 | -4.6 | |
| THINKBRAKE-pBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE-p2025.10 | 62.5 | -28.5 | |
| THINKBRAKEBase Model=Qwen3-4B, Stopping Strategy=THINKBRAKE2025.10 | 62.5 | -21.1 | |
| Qwen3-14BBase Model=Qwen3-14B, Stopping Strategy=Base2025.10 | 62.5 | — | |
| NoWaitBase Model=Qwen3-14B, Stopping Strategy=NoWait2025.10 | 62.5 | -30.1 | |
| THINKBRAKEBase Model=Qwen3-14B, Stopping Strategy=THINKBRAKE2025.10 | 62.5 | -25.4 | |
| NoWaitBase Model=DeepSeek-R1-7B, Stopping Strategy=NoWait2025.10 | 50 | -8.9 | |
| DeepSeek-R1-7BBase Model=DeepSeek-R1-7B, Stopping Strategy=Base2025.10 | 43.8 | — | |
| THINKBRAKEBase Model=DeepSeek-R1-7B, Stopping Strategy=THINKBRAKE2025.10 | 43.8 | -19.9 | |
| ThinkLessBase Model=Qwen3-4B-Thinking, Stopping Strategy=ThinkLess2025.10 | 37.5 | -100 |