Tool Invocation Refusal on SABEval (D3)
85.56Tool Invocation Rate (TIR)Base
Evaluation results
| Method | Links | |
|---|---|---|
| BaseModel=Qwen3-4B2026.04 | 85.56 | |
| BaseModel=Qwen3-14B2026.04 | 83.9 | |
| BaseModel=Qwen3-8B2026.04 | 75.56 | |
| PromptModel=Qwen3-4B2026.04 | 69.71 | |
| PromptModel=ToolACE-2.52026.04 | 67.71 | |
| BaseModel=ToolACE-2.52026.04 | 67.41 | |
| PromptModel=Qwen3-8B2026.04 | 57.66 | |
| PromptModel=Qwen3-14B2026.04 | 56.78 | |
| BaseModel=Watt-Tool2026.04 | 31.51 | |
| PromptModel=Watt-Tool2026.04 | 29.61 | |
| CAAModel=Qwen3-4B2026.04 | 27.76 | |
| CAAModel=Qwen3-14B2026.04 | 15.51 | |
| CAAModel=Qwen3-8B2026.04 | 8.98 | |
| CAAModel=Watt-Tool2026.04 | 8.1 | |
| CAAModel=ToolACE-2.52026.04 | 6.88 |