Tool Usage on BFCL Multi-Parallel v2
87.5AccuracyTHINKBRAKE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| THINKBRAKEBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE2025.10 | 87.5 | -18.9 | |
| Qwen3-4B-ThinkingBase Model=Qwen3-4B-Thinking, Stopping Strategy=Base2025.10 | 79.2 | — | |
| NoWaitBase Model=Qwen3-4B-Thinking, Stopping Strategy=NoWait2025.10 | 75 | -66.5 | |
| NoWaitBase Model=Qwen3-4B, Stopping Strategy=NoWait2025.10 | 75 | -11.1 | |
| THINKBRAKEBase Model=Qwen3-4B, Stopping Strategy=THINKBRAKE2025.10 | 75 | -10.2 | |
| Qwen3-4BBase Model=Qwen3-4B, Stopping Strategy=Base2025.10 | 70.8 | — | |
| Qwen3-14BBase Model=Qwen3-14B, Stopping Strategy=Base2025.10 | 70.8 | — | |
| Qwen3-32BBase Model=Qwen3-32B, Stopping Strategy=Base2025.10 | 70.8 | — | |
| THINKBRAKEBase Model=Phi-4-Reasoning, Stopping Strategy=THINKBRAKE2025.10 | 70.8 | -4.9 | |
| NoWaitBase Model=Qwen3-14B, Stopping Strategy=NoWait2025.10 | 62.5 | -10.9 | |
| THINKBRAKEBase Model=Qwen3-14B, Stopping Strategy=THINKBRAKE2025.10 | 62.5 | -7.6 | |
| NoWaitBase Model=Qwen3-32B, Stopping Strategy=NoWait2025.10 | 62.5 | -14.3 | |
| Phi-4-ReasoningBase Model=Phi-4-Reasoning, Stopping Strategy=Base2025.10 | 58.3 | — | |
| NoWaitBase Model=Phi-4-Reasoning, Stopping Strategy=NoWait2025.10 | 58.3 | -11.9 | |
| THINKBRAKEBase Model=Qwen3-32B, Stopping Strategy=THINKBRAKE2025.10 | 58 | -13.3 | |
| ThinkLessBase Model=Qwen3-4B-Thinking, Stopping Strategy=ThinkLess2025.10 | 45.8 | -100 | |
| THINKBRAKE-pBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE-p2025.10 | 29.2 | -43.8 | |
| DeepSeek-R1-7BBase Model=DeepSeek-R1-7B, Stopping Strategy=Base2025.10 | 29.2 | — | |
| NoWaitBase Model=DeepSeek-R1-7B, Stopping Strategy=NoWait2025.10 | 20.8 | 11 | |
| THINKBRAKEBase Model=DeepSeek-R1-7B, Stopping Strategy=THINKBRAKE2025.10 | 16.7 | -6 |