ToolBench
Benchmarks
TaskDatasetSOTA resultTrendResults
ToolBench
99.61Average Success Rate (ASR)
62
ToolBench
80.67Average Pass Rate
53
ToolBench
58.54NDCG@10
44
ToolBench Average
75.95Pass Rate
30
ToolBench In-domain I1
93.76NDCG@1
29
ToolBench
88API Success Rate
28
ToolBench (test)
83.7Pass@1
28
ToolBench-60 one-hot preference regime (held-out pool)
103.3Regret
27
ToolBench
42.6EM (%)
24
ToolBench
98CDA
24
ToolBench (G3)
91.8Pass Rate
24
ToolBench G2
93Pass Rate
24
ToolBench (G1)
85.5Pass Rate
24
ToolBench G3
71.5Pass Rate
22
ToolBench G2
78.2Pass Rate
22
ToolBench G1 v1
83.5Pass Rate
22
ToolBench In-domain (I3)
91.74NDCG@1
20
ToolBench In-domain (I2)
91.91NDCG@1
20
ToolBench-60 soft-0.3 preference regime (held-out pool)
255.9Regret
18
ToolBench
5.6Energy (Wh)
18
ToolBench
4,602Throughput (tokens/s)
18
ToolBench
3.9Goodput (req/s)
18
ToolBench
44.98Success Rate
16
ToolBench I1 v1
56.13SoPR
16
ToolBench I3-Inst
52.4Pass Rate
14