API-Bank
Benchmarks
TaskDatasetSOTA resultTrendResults
API-Bank L-1
94.99F1 Name Match
46
API-Bank (test)
74Success Rate
28
API-Bank L-2
90.42Name Match F1
25
API-Bank
5.8Match Rate (#MAT)
22
API-Bank
79.17Level-1 Score
20
API-Bank (test)
75.54Overall Accuracy
19
API-Bank
77.19Success Rate
18
API-Bank Level 2
66.22Accuracy
18
API-Bank
79.1Success Rate
12
API-Bank L-2 v1 (test)
88F1 Name Match
12
API-Bank L-1 v1 (test)
90.78F1 Score
12
API-Bank Level-2
83.2ROUGE-L
12
API-Bank Level-1
93.4ROUGE-L
12
API Bank
90Accuracy
10
API-Bank LV2
62.41Correctness
10
API-Bank reconstructed
79.27Correctness
9
API-Bank
90.45Accuracy
8
API-Bank Call+Retrieve
26.9Task Completion Rate
8
API-Bank Call
34.7Task Completion Rate
8
API-Bank Level-3
90.67Recall@k
7
API-Bank (L1)
81.3Score
6
API-Bank Level-3 50 instances (LOO-CV)
94.5Set F1
6
API-Bank L2 cleaned (test)
87.32F1 (API Matching)
5
API-Bank (test)
59.12Recall@1
4