Tool-Calling Decision Making on When2Call (test)
80.83Normalized AccuracyTurn-level TRUST
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Turn-level TRUSTModel Base=4B-Thinking, Training=Turn-level2026.06 | 80.83 | 82.84 | 17.83 | 5.07 | |
| Claude-Sonnet-4Model Base=-, Training=-2026.06 | 80.53 | 81.85 | 10.08 | 5.86 | |
| SAGEModel Base=4B-Thinking, Training=Training-free2026.06 | 73.36 | 72.36 | 0 | 8.89 | |
| GRPOModel Base=4B-Thinking, Training=Turn-level2026.06 | 72.46 | 76.06 | 5.73 | 24.76 | |
| SAGEModel Base=8B-Thinking, Training=Training-free2026.06 | 71.87 | 71.57 | 0 | 11.04 | |
| 4B-ThinkingModel Base=-, Training=-2026.06 | 69.36 | 72.77 | 41.47 | 17.11 | |
| Traj.-level TRUSTModel Base=8B-Base, Training=Traj.-level2026.06 | 62.32 | 60.62 | 49.87 | 8.39 | |
| GPT-4o-miniModel Base=-, Training=-2026.06 | 57.64 | 60.26 | 59.3 | 11.61 | |
| AUQModel Base=4B-Thinking, Training=Training-free2026.06 | 56.35 | 60.32 | 45.71 | 22.98 | |
| 8B-ThinkingModel Base=-, Training=-2026.06 | 45.19 | 50.3 | 14.73 | 32.78 | |
| Traj.-level CM2Model Base=8B-Base, Training=Traj.-level2026.06 | 43.75 | 36.94 | 75.43 | 9.34 | |
| 235B-A22B-InstructModel Base=-, Training=-2026.06 | 42.33 | 32.62 | 82.95 | 4.63 | |
| 30B-A3B-InstructModel Base=-, Training=-2026.06 | 35.1 | 17.75 | 94.57 | 1.34 | |
| AUQModel Base=8B-Thinking, Training=Training-free2026.06 | 34.34 | 37.91 | 37.21 | 36.99 | |
| MiniMax-M2.5Model Base=-, Training=-2026.06 | 23.93 | 24.94 | 24.03 | 20.48 |