ToolBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ToolBench generalization dataset (I1-Tool)
57.7SoPR
7
ToolBench I3 conditional attack setting
70.51Hit@1
6
ToolBench I1
2.686Hit@1
6
ToolBench Multi-Domain
86.07NDCG@1 (I1)
6
ToolBench With Retriever (End-to-End)
59.2SoPR (I1)
6
ToolBench With Ground-Truth Tools
60.7SoPR (I1)
6
ToolBench (TB)
0.82Krippendorff's Alpha
6
ToolBench standard evaluation
51.3Pass Rate
6
ToolBench
71.6Pass@1
5
ToolBench G3 (test)
25.8Recall@1
4
ToolBench Category Gen
55.95SoPR
4
ToolBench Tool Gen
52.2SoPR
4
ToolBench I3
61.77NDCG@5
4
ToolBench I2
75.37NDCG@5
4
ToolBench I1
84.16NDCG@5
4
ToolBench I2-Cat v1
46.24SoPR
4
ToolBench
92.6Attack Hit Rate (AHR)
3
ToolBench Async subset G3
0Async Rate
3
ToolBench (test)
91TDR
3
ToolBench
61.9BoR Found Rate
1
ToolBench (test)
28.3DiffLM Win Rate
1