Tool Retrieval on ToolBench I3 (test)
76.63Recall@3COLT
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| COLTBackbone=Contriever, Framework=+COLT (Ours)2024.05 | 76.63 | 85.5 | 81.21 | 84.18 | 52 | 68.47 | |
| COLTBackbone=coCondensor, Framework=+COLT (Ours)2024.05 | 75.48 | 84.97 | 80 | 83.55 | 49.17 | 68.64 | |
| COLTBackbone=TAS-B, Framework=+COLT (Ours)2024.05 | 74.49 | 84.58 | 79.03 | 82.95 | 48.16 | 68.35 | |
| COLTBackbone=ANCE, Framework=+COLT (Ours)2024.05 | 73.37 | 83.97 | 77.95 | 82.14 | 46.01 | 66.41 | |
| ContrieverFramework=+Fine-tune2024.05 | 68.58 | 80.05 | 72.86 | 77.69 | 39.7 | 60.89 | |
| coCondensorFramework=+Fine-tune2024.05 | 66.97 | 79.3 | 71.2 | 76.5 | 37.08 | 58.66 | |
| TAS-BFramework=+Fine-tune2024.05 | 66.04 | 77.64 | 70.41 | 75.34 | 35.69 | 55.75 | |
| ANCEFramework=+Fine-tune2024.05 | 65.11 | 76.63 | 69.27 | 74.14 | 34.68 | 53.64 | |
| ContrieverFramework=Zero-shot2024.05 | 31.37 | 38.6 | 34.13 | 37.37 | 6.03 | 11.42 | |
| BM25Backbone=BM252024.05 | 29.33 | 35.88 | 32.2 | 35.08 | 5.52 | 9.78 | |
| TAS-BFramework=Zero-shot2024.05 | 25.32 | 31.15 | 27.8 | 30.36 | 3.84 | 6.4 | |
| ANCEFramework=Zero-shot2024.05 | 21.55 | 26.38 | 23.44 | 25.6 | 2.44 | 4.59 | |
| coCondensorFramework=Zero-shot2024.05 | 20.8 | 25.24 | 23.21 | 25.1 | 2.07 | 3.75 |