# Tool Use on ToolHop Source: [Tool Use on ToolHop]() Collection index: [llms.txt]() Data updated: 2026-06-02T12:07:20.832Z Evaluation of general tool utilization capabilities on the ToolHop dataset, measured by Answer Correctness (Ans.) and Path Success Rate (Path). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Tool Use | | Dataset (including any stated split) | ToolHop | | Metric | Answer Correctness | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Answer Correctness](): Higher is better; unit: PERCENT. - [Path Success Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Answer Correctness | Result date | Report paper | | --- | --- | --- | --- | | HarnessForge | 54.87 | Not recorded | 6a1e6435dbd7150506f8583d | | HarnessForge | 52.82 | Not recorded | 6a1e6435dbd7150506f8583d | | GRPO | 51.28 | Not recorded | 6a1e6435dbd7150506f8583d | | RLOO | 50.77 | Not recorded | 6a1e6435dbd7150506f8583d | | GRPO | 49.74 | Not recorded | 6a1e6435dbd7150506f8583d | | SFT | 48.72 | Not recorded | 6a1e6435dbd7150506f8583d | | MermaidFlow | 47.39 | Not recorded | 6a1e6435dbd7150506f8583d | | RLOO | 46.15 | Not recorded | 6a1e6435dbd7150506f8583d | | SFT | 45.13 | Not recorded | 6a1e6435dbd7150506f8583d | | MermaidFlow | 44.62 | Not recorded | 6a1e6435dbd7150506f8583d | | MaAS | 44.62 | Not recorded | 6a1e6435dbd7150506f8583d | | MaAS | 42.05 | Not recorded | 6a1e6435dbd7150506f8583d | | ADAS | 42.05 | Not recorded | 6a1e6435dbd7150506f8583d | | ADAS | 40 | Not recorded | 6a1e6435dbd7150506f8583d | | AFlow | 32.82 | Not recorded | 6a1e6435dbd7150506f8583d | | AFlow | 31.28 | Not recorded | 6a1e6435dbd7150506f8583d | | AgentSquare | 30.77 | Not recorded | 6a1e6435dbd7150506f8583d | | AgentSquare | 29.23 | Not recorded | 6a1e6435dbd7150506f8583d | ### HarnessForge Answer Correctness: 54.87. Result ID: 6a1ec734fbc373592c1a4a9f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### HarnessForge Answer Correctness: 52.82. Result ID: 6a1ec734fbc373592c1a4a96. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### GRPO Answer Correctness: 51.28. Result ID: 6a1ec734fbc373592c1a4a9e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### RLOO Answer Correctness: 50.77. Result ID: 6a1ec734fbc373592c1a4a9d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### GRPO Answer Correctness: 49.74. Result ID: 6a1ec734fbc373592c1a4a95. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### SFT Answer Correctness: 48.72. Result ID: 6a1ec734fbc373592c1a4a9c. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### MermaidFlow Answer Correctness: 47.39. Result ID: 6a1ec734fbc373592c1a4a9b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### RLOO Answer Correctness: 46.15. Result ID: 6a1ec734fbc373592c1a4a94. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### SFT Answer Correctness: 45.13. Result ID: 6a1ec734fbc373592c1a4a93. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### MermaidFlow Answer Correctness: 44.62. Result ID: 6a1ec734fbc373592c1a4a92. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### MaAS Answer Correctness: 44.62. Result ID: 6a1ec734fbc373592c1a4a9a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### MaAS Answer Correctness: 42.05. Result ID: 6a1ec734fbc373592c1a4a91. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### ADAS Answer Correctness: 42.05. Result ID: 6a1ec734fbc373592c1a4a97. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### ADAS Answer Correctness: 40. Result ID: 6a1ec734fbc373592c1a4a8e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### AFlow Answer Correctness: 32.82. Result ID: 6a1ec734fbc373592c1a4a99. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### AFlow Answer Correctness: 31.28. Result ID: 6a1ec734fbc373592c1a4a90. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. ### AgentSquare Answer Correctness: 30.77. Result ID: 6a1ec734fbc373592c1a4a98. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction. ### AgentSquare Answer Correctness: 29.23. Result ID: 6a1ec734fbc373592c1a4a8f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.