Tool Retrieval on ToolBench In-Domain
86.07NDCG@1 (I1)ToolOmni
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| ToolOmniBackbone=Qwen3-4B-Instruct, Maximum retrieval turns=42026.04 | 86.07 | 84.49 | 85.56 | 81.5 | 73.13 | 74.11 | 84.4 | 76.81 | 77.25 | 80.37 | |
| ToolRetrieverMechanism=Dense retrieval2026.04 | 81.91 | 82.05 | 85.57 | 75.92 | 69.61 | 75.4 | 79.82 | 72.42 | 77.11 | 77.76 | |
| IterFeedbackMaximum retrieval turns=42026.04 | 71.64 | 71.29 | 76.31 | 62.65 | 55.58 | 60.62 | 73.85 | 64.06 | 69.02 | 67.22 | |
| Re-Invoke*Source=Cited from ToolGen2026.04 | 69.47 | — | 61.1 | 54.56 | — | 53.79 | 59.65 | — | 59.55 | 59.69 | |
| ToolGenMechanism=Generative retrieval2026.04 | 69.47 | 72.26 | 79.12 | 46.77 | 53.58 | 62.45 | 77.06 | 76.44 | 85.48 | 69.18 | |
| EmbSim*Backbone=text-embedding-3-large, Source=Cited from ToolGen2026.04 | 63.67 | 61.03 | 65.37 | 49.11 | 42.27 | 46.56 | 53 | 46.4 | 52.73 | 53.35 | |
| BM25*Source=Cited from ToolGen2026.04 | 29.46 | 31.12 | 33.27 | 24.13 | 25.29 | 27.65 | 32 | 25.88 | 29.78 | 28.73 |