Tool Retrieval on ToolBench I2 (test)
75.72Recall@3COLT
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| COLTBackbone=Contriever, Framework=+COLT (Ours)2024.05 | 75.72 | 85.03 | 78.57 | 82.54 | 51.97 | 70.1 | |
| COLTBackbone=coCondensor, Framework=+COLT (Ours)2024.05 | 73.91 | 83.47 | 76.75 | 80.87 | 49.15 | 67.75 | |
| COLTBackbone=TAS-B, Framework=+COLT (Ours)2024.05 | 71.64 | 81.12 | 74.6 | 78.74 | 46.77 | 64.38 | |
| COLTBackbone=ANCE, Framework=+COLT (Ours)2024.05 | 70.76 | 80.59 | 73.64 | 77.98 | 45.1 | 62.93 | |
| TAS-BFramework=+Fine-tune2024.05 | 62.78 | 67.49 | 58.96 | 64.21 | 26.74 | 43.66 | |
| ContrieverFramework=+Fine-tune2024.05 | 58.89 | 70.75 | 62.11 | 67.42 | 29.77 | 48.31 | |
| ANCEFramework=+Fine-tune2024.05 | 58.58 | 67.2 | 58.58 | 63.75 | 26.46 | 42.8 | |
| coCondensorFramework=+Fine-tune2024.05 | 57.7 | 69.46 | 60.8 | 66.07 | 28.78 | 46.06 | |
| ContrieverFramework=Zero-shot2024.05 | 25.67 | 31.15 | 26.96 | 29.95 | 7.46 | 9.8 | |
| ANCEFramework=Zero-shot2024.05 | 20.82 | 26.56 | 21.45 | 24.57 | 5.06 | 7.46 | |
| TAS-BFramework=Zero-shot2024.05 | 19.1 | 23.71 | 19.81 | 22.33 | 5.17 | 7.14 | |
| BM25Backbone=BM252024.05 | 17.06 | 21.38 | 17.83 | 19.88 | 2.39 | 4.37 | |
| coCondensorFramework=Zero-shot2024.05 | 15.33 | 19.37 | 16.15 | 18.32 | 3.02 | 5.3 |