Tool Sequence Recommendation on API-Bank Level-3 50 instances (LOO-CV)
94.5Set F1LLaMA-3.1-8B
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLaMA-3.1-8Bn-shot=3-shot, Evaluation protocol=LOO-CV, Candidate set=oracle candidate set (ground-truth tools)2026.04 | 94.5 | 74.7 | — | — | |
| Hybrid Sem-GraphEvaluation protocol=LOO-CV2026.04 | 80.3 | 19.3 | 0.293 | 19 | |
| GS-Hybrid + Hyb-RerankEvaluation protocol=LOO-CV, Stage-1=GS-Hybrid, Reranker=Hyb-Rerank2026.04 | 80.3 | 17.3 | 0.333 | 19 | |
| GS-Hybrid + LREvaluation protocol=LOO-CV, Stage-1=GS-Hybrid, Reranker=Learned Reranker, Training Mode=Zero-shot (reused ToolBench-trained weights)2026.04 | 80.3 | 64.7 | 0.613 | 62 | |
| Semantic OnlyEvaluation protocol=LOO-CV2026.04 | 79.3 | 13.3 | 0.433 | 15 | |
| BM25Evaluation protocol=LOO-CV2026.04 | 59 | 10 | 0.1 | 10 |