ToolBench
Benchmarks
TaskDatasetSOTA resultTrendResults
ToolBench v1 (I2)
52.2SoPR
12
ToolBench I1-Cat v1
61.76SoPR
11
ToolBench
0.813AUROC
10
TOOLBENCH-60 soft-0.3 preference regime
264.8Regret
9
ToolBench dataset
100SLO Attainment
9
ToolBench
1.11Effective Throughput (req/s)
9
ToolBench I3
88NDCG@1
9
ToolBench Multi-domain (I2)
89.76NDCG@1
9
ToolBench Vague queries G3 tier
21.1NDCG@5
8
ToolBench Vague queries, G2 tier
38.7NDCG@5
8
ToolBench Vague queries G1 tier
49.4NDCG@5
8
ToolBench Standard queries, G3 tier
60.1NDCG@5
8
ToolBench Standard queries G2 tier
73.6NDCG@5
8
ToolBench Standard queries, G1 tier
86.8NDCG@5
8
ToolBench (test)
27.1Set-F1
8
ToolBench Out of the domain
83.9Plan Accuracy
8
ToolBench (In the domain)
82.5Plan ACC
8
ToolBench
37.8Task Completion Rate (TCR)
8
ToolBench
0.74Pearson r
8
ToolBench
76.1TAFC Win Rate
8
ToolBench In-Domain
86.07NDCG@1 (I1)
7
ToolBench 30% 2,413 tools, 180 queries (test)
39.2Recall@1
7
ToolBench v1 (Evaluation)
71.7Success Rate
7
ToolBench
97Precision@R
7
ToolBench generalization dataset (I2-Cat)
51.96SoPR
7