# Tool Learning under Instructions Beyond Tool Capabilities on NoisyToolBench IBTC 1.0 (test) Source: [Tool Learning under Instructions Beyond Tool Capabilities on NoisyToolBench IBTC 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-04-30T06:42:20.997Z Evaluation of model performance on tool learning capabilities when instructions require reasoning beyond the explicit capabilities of the provided tools, using the test split of NoisyToolBench IBTC version 1.0. Results are reported using the A1 metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Tool Learning under Instructions Beyond Tool Capabilities | | Dataset (including any stated split) | NoisyToolBench IBTC 1.0 (test) | | Metric | A1 Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [A1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | A1 Score | Result date | Report paper | | --- | --- | --- | --- | | CoT + AwN | 98 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 98 | Not recorded | 68fa77061e209131cea7092e | | CoT + AwN | 94 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 94 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 94 | Not recorded | 68fa77061e209131cea7092e | | CoT + AwN | 92 | Not recorded | 68fa77061e209131cea7092e | | CoT + AwN | 88 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 86 | Not recorded | 68fa77061e209131cea7092e | | CoT + AwN | 84 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 76 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 70 | Not recorded | 68fa77061e209131cea7092e | | CoT + AwN | 68 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 64 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 62 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 54 | Not recorded | 68fa77061e209131cea7092e | | CoT | 52 | Not recorded | 68fa77061e209131cea7092e | | CoT + AwN | 48 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 48 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 48 | Not recorded | 68fa77061e209131cea7092e | | DFSDT + AwN | 48 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 46 | Not recorded | 68fa77061e209131cea7092e | | CoT + AwN | 40 | Not recorded | 68fa77061e209131cea7092e | | CoT | 38 | Not recorded | 68fa77061e209131cea7092e | | CoT | 34 | Not recorded | 68fa77061e209131cea7092e | | CoT | 30 | Not recorded | 68fa77061e209131cea7092e | | CoT | 30 | Not recorded | 68fa77061e209131cea7092e | | CoT | 10 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 8 | Not recorded | 68fa77061e209131cea7092e | | CoT | 0 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 0 | Not recorded | 68fa77061e209131cea7092e | | CoT | 0 | Not recorded | 68fa77061e209131cea7092e | | DFSDT | 0 | Not recorded | 68fa77061e209131cea7092e | ### CoT + AwN A1 Score: 98. Result ID: 69f2f91c47f34ce924d8bd4b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"deepseek-v3"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 98. Result ID: 69f2f91c47f34ce924d8bd4d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"deepseek-v3"}. This record does not assert independent reproduction. ### CoT + AwN A1 Score: 94. Result ID: 69f2f91c47f34ce924d8bd43. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-4"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 94. Result ID: 69f2f91c47f34ce924d8bd45. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-4"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 94. Result ID: 69f2f91c47f34ce924d8bd49. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-4o"}. This record does not assert independent reproduction. ### CoT + AwN A1 Score: 92. Result ID: 69f2f91c47f34ce924d8bd47. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-4o"}. This record does not assert independent reproduction. ### CoT + AwN A1 Score: 88. Result ID: 69f2f91c47f34ce924d8bd53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-3.5"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 86. Result ID: 69f2f91c47f34ce924d8bd55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-3.5"}. This record does not assert independent reproduction. ### CoT + AwN A1 Score: 84. Result ID: 69f2f91c47f34ce924d8bd5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-R1"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 76. Result ID: 69f2f91c47f34ce924d8bd5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-R1"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 70. Result ID: 69f2f91c47f34ce924d8bd51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gemini-1.5"}. This record does not assert independent reproduction. ### CoT + AwN A1 Score: 68. Result ID: 69f2f91c47f34ce924d8bd4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gemini-1.5"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 64. Result ID: 69f2f91c47f34ce924d8bd41. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-3.5"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 62. Result ID: 69f2f91c47f34ce924d8bd54. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"claude-3.5"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 54. Result ID: 69f2f91c47f34ce924d8bd44. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-4"}. This record does not assert independent reproduction. ### CoT A1 Score: 52. Result ID: 69f2f91c47f34ce924d8bd4e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gemini-1.5"}. This record does not assert independent reproduction. ### CoT + AwN A1 Score: 48. Result ID: 69f2f91c47f34ce924d8bd3f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-3.5"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 48. Result ID: 69f2f91c47f34ce924d8bd4c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"deepseek-v3"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 48. Result ID: 69f2f91c47f34ce924d8bd50. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gemini-1.5"}. This record does not assert independent reproduction. ### DFSDT + AwN A1 Score: 48. Result ID: 69f2f91c47f34ce924d8bd59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"O3-mini"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 46. Result ID: 69f2f91c47f34ce924d8bd40. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-3.5"}. This record does not assert independent reproduction. ### CoT + AwN A1 Score: 40. Result ID: 69f2f91c47f34ce924d8bd57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"O3-mini"}. This record does not assert independent reproduction. ### CoT A1 Score: 38. Result ID: 69f2f91c47f34ce924d8bd3e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-3.5"}. This record does not assert independent reproduction. ### CoT A1 Score: 34. Result ID: 69f2f91c47f34ce924d8bd42. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-4"}. This record does not assert independent reproduction. ### CoT A1 Score: 30. Result ID: 69f2f91c47f34ce924d8bd4a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"deepseek-v3"}. This record does not assert independent reproduction. ### CoT A1 Score: 30. Result ID: 69f2f91c47f34ce924d8bd52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-3.5"}. This record does not assert independent reproduction. ### CoT A1 Score: 10. Result ID: 69f2f91c47f34ce924d8bd46. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"gpt-4o"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 8. Result ID: 69f2f91c47f34ce924d8bd48. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"gpt-4o"}. This record does not assert independent reproduction. ### CoT A1 Score: 0. Result ID: 69f2f91c47f34ce924d8bd56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"O3-mini"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 0. Result ID: 69f2f91c47f34ce924d8bd58. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"O3-mini"}. This record does not assert independent reproduction. ### CoT A1 Score: 0. Result ID: 69f2f91c47f34ce924d8bd5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"DeepSeek-R1"}. This record does not assert independent reproduction. ### DFSDT A1 Score: 0. Result ID: 69f2f91c47f34ce924d8bd5c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"DeepSeek-R1"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.