Tool Usage on Aggregate BFCL and Meta-Tool
86.6AccuracyTHINKBRAKE
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| THINKBRAKEBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE2025.10 | 86.6 | -22.6 | |
| Qwen3-4B-ThinkingBase Model=Qwen3-4B-Thinking, Stopping Strategy=Base2025.10 | 84.3 | — | |
| Qwen3-4BBase Model=Qwen3-4B, Stopping Strategy=Base2025.10 | 81.6 | — | |
| NoWaitBase Model=Qwen3-4B-Thinking, Stopping Strategy=NoWait2025.10 | 81.2 | -37.7 | |
| NoWaitBase Model=Qwen3-4B, Stopping Strategy=NoWait2025.10 | 80.8 | -6.7 | |
| THINKBRAKEBase Model=Phi-4-Reasoning, Stopping Strategy=THINKBRAKE2025.10 | 79.2 | -2 | |
| THINKBRAKEBase Model=Qwen3-4B, Stopping Strategy=THINKBRAKE2025.10 | 79 | -10.7 | |
| Phi-4-ReasoningBase Model=Phi-4-Reasoning, Stopping Strategy=Base2025.10 | 78.8 | — | |
| Qwen3-32BBase Model=Qwen3-32B, Stopping Strategy=Base2025.10 | 77.8 | — | |
| NoWaitBase Model=Phi-4-Reasoning, Stopping Strategy=NoWait2025.10 | 77.2 | -9.5 | |
| THINKBRAKEBase Model=Qwen3-32B, Stopping Strategy=THINKBRAKE2025.10 | 76.3 | -7 | |
| Qwen3-14BBase Model=Qwen3-14B, Stopping Strategy=Base2025.10 | 76.2 | — | |
| THINKBRAKEBase Model=Qwen3-14B, Stopping Strategy=THINKBRAKE2025.10 | 75.8 | -12.4 | |
| NoWaitBase Model=Qwen3-32B, Stopping Strategy=NoWait2025.10 | 75.4 | -5.2 | |
| NoWaitBase Model=Qwen3-14B, Stopping Strategy=NoWait2025.10 | 74.1 | -10.6 | |
| ThinkLessBase Model=Qwen3-4B-Thinking, Stopping Strategy=ThinkLess2025.10 | 66.6 | -100 | |
| THINKBRAKE-pBase Model=Qwen3-4B-Thinking, Stopping Strategy=THINKBRAKE-p2025.10 | 61.6 | -28.5 | |
| NoWaitBase Model=DeepSeek-R1-7B, Stopping Strategy=NoWait2025.10 | 52.2 | -4.6 | |
| DeepSeek-R1-7BBase Model=DeepSeek-R1-7B, Stopping Strategy=Base2025.10 | 51.2 | — | |
| THINKBRAKEBase Model=DeepSeek-R1-7B, Stopping Strategy=THINKBRAKE2025.10 | 46.5 | -14.3 |