Tool Invocation Refusal on SABEval (D2)
5.46Tool Invocation Rate (TIR)CAA
Evaluation results
| Method | Links | |
|---|---|---|
| CAAModel=Watt-Tool2026.04 | 5.46 | |
| CAAModel=ToolACE-2.52026.04 | 5.76 | |
| CAAModel=Qwen3-8B2026.04 | 7.27 | |
| CAAModel=Qwen3-14B2026.04 | 9.31 | |
| CAAModel=Qwen3-4B2026.04 | 20.29 | |
| PromptModel=Watt-Tool2026.04 | 22.34 | |
| BaseModel=Watt-Tool2026.04 | 23.31 | |
| PromptModel=Qwen3-14B2026.04 | 43.56 | |
| PromptModel=Qwen3-8B2026.04 | 48.73 | |
| PromptModel=Qwen3-4B2026.04 | 56.83 | |
| PromptModel=ToolACE-2.52026.04 | 62.63 | |
| BaseModel=ToolACE-2.52026.04 | 63.02 | |
| BaseModel=Qwen3-8B2026.04 | 66.29 | |
| BaseModel=Qwen3-4B2026.04 | 73.41 | |
| BaseModel=Qwen3-14B2026.04 | 74.73 |