# Tool-use Reasoning on Ours (test) Source: [Tool-use Reasoning on Ours (test)]() Collection index: [llms.txt]() Data updated: 2026-04-16T07:03:06.287Z Evaluation of model performance on Tool-use Reasoning capabilities using the proprietary 'Ours (test)' dataset, measured using Solve Precision (P), Recall (R), and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Tool-use Reasoning | | Dataset (including any stated split) | Ours (test) | | Metric | Solve Precision (P) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Solve Precision (P)](): Higher is better; unit: PERCENT. - [Solve Recall (R)](): Higher is better; unit: PERCENT. - [Solve F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Solve Precision (P) | Result date | Report paper | | --- | --- | --- | --- | | Gemini 2.5-Flash | 52.78 | Not recorded | 68fa7b401e209131cea874c8 | | Gemini 2.5-Pro | 51.77 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-14B FTRL-Reinforce++ | 49.51 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Non-Reasoning FTRL-GRPO | 49.41 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-14B FTRL-GRPO | 48.67 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-7B FTRL-GRPO | 47.82 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Non-Reasoning FTRL-GRPO | 46.44 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Reasoning FTRL-Reinforce++ | 44.15 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Non-Reasoning FTRL-Reinforce++ | 44.06 | Not recorded | 68fa7b401e209131cea874c8 | | Claude 4.0-Sonnet | 42.33 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Non-Reasoning FTRL-Reinforce++ | 41.96 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Reasoning FTRL-GRPO | 41.67 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-7B FTRL-Reinforce++ | 41.61 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Reasoning FTRL-GRPO | 40.67 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Reasoning FTRL-Reinforce++ | 38.11 | Not recorded | 68fa7b401e209131cea874c8 | | GPT 4o | 33.43 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-14B Instruct | 33.31 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-32B Non-Reasoning | 33.2 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Reasoning | 31.91 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-72B Instruct | 31.68 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-32B Instruct | 30.12 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-32B Reasoning | 28.94 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Non-Reasoning | 28.29 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Reasoning | 27.78 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-7B Instruct | 27.44 | Not recorded | 68fa7b401e209131cea874c8 | | GPT 3.5-Turbo | 25.77 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Non-Reasoning | 21.18 | Not recorded | 68fa7b401e209131cea874c8 | ### Gemini 2.5-Flash Solve Precision (P): 52.78. Result ID: 69e0881e47f34ce924c94353. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Version":"2.5-Flash"}. This record does not assert independent reproduction. ### Gemini 2.5-Pro Solve Precision (P): 51.77. Result ID: 69e0881e47f34ce924c94354. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Version":"2.5-Pro"}. This record does not assert independent reproduction. ### Qwen2.5-14B FTRL-Reinforce++ Solve Precision (P): 49.51. Result ID: 69e0881e47f34ce924c94360. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-14B","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen3-14B Non-Reasoning FTRL-GRPO Solve Precision (P): 49.41. Result ID: 69e0881e47f34ce924c94367. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Non-Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen2.5-14B FTRL-GRPO Solve Precision (P): 48.67. Result ID: 69e0881e47f34ce924c94361. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-14B","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen2.5-7B FTRL-GRPO Solve Precision (P): 47.82. Result ID: 69e0881e47f34ce924c9435e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-7B","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen3-8B Non-Reasoning FTRL-GRPO Solve Precision (P): 46.44. Result ID: 69e0881e47f34ce924c94364. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Non-Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen3-14B Reasoning FTRL-Reinforce++ Solve Precision (P): 44.15. Result ID: 69e0881e47f34ce924c9436c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen3-14B Non-Reasoning FTRL-Reinforce++ Solve Precision (P): 44.06. Result ID: 69e0881e47f34ce924c94366. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Non-Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Claude 4.0-Sonnet Solve Precision (P): 42.33. Result ID: 69e0881e47f34ce924c94355. [Reporting paper]() Recorded evaluation configuration: {"Version":"4.0-Sonnet"}. This record does not assert independent reproduction. ### Qwen3-8B Non-Reasoning FTRL-Reinforce++ Solve Precision (P): 41.96. Result ID: 69e0881e47f34ce924c94363. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Non-Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen3-14B Reasoning FTRL-GRPO Solve Precision (P): 41.67. Result ID: 69e0881e47f34ce924c9436d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen2.5-7B FTRL-Reinforce++ Solve Precision (P): 41.61. Result ID: 69e0881e47f34ce924c9435d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-7B","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen3-8B Reasoning FTRL-GRPO Solve Precision (P): 40.67. Result ID: 69e0881e47f34ce924c9436a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen3-8B Reasoning FTRL-Reinforce++ Solve Precision (P): 38.11. Result ID: 69e0881e47f34ce924c94369. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### GPT 4o Solve Precision (P): 33.43. Result ID: 69e0881e47f34ce924c94357. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Version":"4o"}. This record does not assert independent reproduction. ### Qwen2.5-14B Instruct Solve Precision (P): 33.31. Result ID: 69e0881e47f34ce924c9435f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-14B","Version":"Instruct"}. This record does not assert independent reproduction. ### Qwen3-32B Non-Reasoning Solve Precision (P): 33.2. Result ID: 69e0881e47f34ce924c9435a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-32B","Version":"Non-Reasoning"}. This record does not assert independent reproduction. ### Qwen3-14B Reasoning Solve Precision (P): 31.91. Result ID: 69e0881e47f34ce924c9436b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Reasoning"}. This record does not assert independent reproduction. ### Qwen2.5-72B Instruct Solve Precision (P): 31.68. Result ID: 69e0881e47f34ce924c94359. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-72B","Version":"Instruct"}. This record does not assert independent reproduction. ### Qwen2.5-32B Instruct Solve Precision (P): 30.12. Result ID: 69e0881e47f34ce924c94358. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-32B","Version":"Instruct"}. This record does not assert independent reproduction. ### Qwen3-32B Reasoning Solve Precision (P): 28.94. Result ID: 69e0881e47f34ce924c9435b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-32B","Version":"Reasoning"}. This record does not assert independent reproduction. ### Qwen3-14B Non-Reasoning Solve Precision (P): 28.29. Result ID: 69e0881e47f34ce924c94365. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Non-Reasoning"}. This record does not assert independent reproduction. ### Qwen3-8B Reasoning Solve Precision (P): 27.78. Result ID: 69e0881e47f34ce924c94368. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Reasoning"}. This record does not assert independent reproduction. ### Qwen2.5-7B Instruct Solve Precision (P): 27.44. Result ID: 69e0881e47f34ce924c9435c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-7B","Version":"Instruct"}. This record does not assert independent reproduction. ### GPT 3.5-Turbo Solve Precision (P): 25.77. Result ID: 69e0881e47f34ce924c94356. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Version":"3.5-Turbo"}. This record does not assert independent reproduction. ### Qwen3-8B Non-Reasoning Solve Precision (P): 21.18. Result ID: 69e0881e47f34ce924c94362. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Non-Reasoning"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.