Tool Calling on WHEN2TOOL Hard
-0.8Delta Accuracy (ΔAcc)PROBE&PREFILL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PROBE&PREFILLmodel averaging=six models, threshold (τ)=0.52026.05 | -0.8 | -0.24 | -3.4 | |
| Necessary (N)model averaging=six models2026.05 | -1.9 | -0.04 | -44.4 | |
| Sparse (S)model averaging=six models2026.05 | -11.1 | -0.35 | -31.5 | |
| Necessary + Reason-then-Actmodel averaging=six models2026.05 | -23.2 | -0.7 | -32.9 | |
| Sparse + Reason-then-Actmodel averaging=six models2026.05 | -29.3 | -0.84 | -34.7 | |
| No Tool + Reason-then-Actmodel averaging=six models2026.05 | -36.6 | -0.94 | -39.1 | |
| No Tool (X)model averaging=six models2026.05 | -41.4 | -0.69 | -60.4 |