Tool Invocation Refusal on SABEval (D1)
59.46Tool Invocation Rate (TIR)Base
Evaluation results
| Method | Links | |
|---|---|---|
| BaseModel=Qwen3-14B2026.04 | 59.46 | |
| BaseModel=Qwen3-4B2026.04 | 57.85 | |
| PromptModel=ToolACE-2.52026.04 | 53.66 | |
| BaseModel=Qwen3-8B2026.04 | 51.61 | |
| BaseModel=ToolACE-2.52026.04 | 51.12 | |
| PromptModel=Qwen3-4B2026.04 | 42.2 | |
| PromptModel=Qwen3-8B2026.04 | 34.29 | |
| PromptModel=Qwen3-14B2026.04 | 33.41 | |
| BaseModel=Watt-Tool2026.04 | 15.46 | |
| PromptModel=Watt-Tool2026.04 | 15.37 | |
| CAAModel=Qwen3-4B2026.04 | 14.29 | |
| CAAModel=Qwen3-14B2026.04 | 7.27 | |
| CAAModel=Qwen3-8B2026.04 | 6.49 | |
| CAAModel=ToolACE-2.52026.04 | 5.27 | |
| CAAModel=Watt-Tool2026.04 | 2.98 |