Tool Invocation Refusal on SABEval 1.0 (D0)
2.54Tool Invocation Rate (TIR)CAA
Evaluation results
| Method | Links | |
|---|---|---|
| CAAModel=Watt-Tool2026.04 | 2.54 | |
| CAAModel=ToolACE-2.52026.04 | 3.41 | |
| CAAModel=Qwen3-8B2026.04 | 4.29 | |
| CAAModel=Qwen3-14B2026.04 | 5.56 | |
| CAAModel=Qwen3-4B2026.04 | 8.29 | |
| PromptModel=Watt-Tool2026.04 | 8.59 | |
| BaseModel=Watt-Tool2026.04 | 8.68 | |
| PromptModel=Qwen3-8B2026.04 | 19.9 | |
| PromptModel=Qwen3-14B2026.04 | 24.29 | |
| PromptModel=Qwen3-4B2026.04 | 27.12 | |
| BaseModel=Qwen3-8B2026.04 | 30.39 | |
| BaseModel=Qwen3-4B2026.04 | 37.61 | |
| BaseModel=ToolACE-2.52026.04 | 37.95 | |
| PromptModel=ToolACE-2.52026.04 | 39.12 | |
| BaseModel=Qwen3-14B2026.04 | 41.37 |