# Robustness of Tool-use on RoTBench Source: [Robustness of Tool-use on RoTBench]() Collection index: [llms.txt]() Data updated: 2026-04-16T07:03:06.340Z Evaluation of agent robustness in tool-using tasks using the RoTBench dataset, reporting metrics such as TS, PI, and CF. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Robustness of Tool-use | | Dataset (including any stated split) | RoTBench | | Metric | TS | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [TS](): Higher is better; unit: PERCENT. - [PI](): Higher is better; unit: PERCENT. - [CF](): Higher is better; unit: PERCENT. ## Recorded results | Method | TS | Result date | Report paper | | --- | --- | --- | --- | | GPT 4o | 78.76 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Non-Reasoning FTRL-Reinforce++ | 78.1 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Non-Reasoning FTRL-Reinforce++ | 77.81 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Non-Reasoning FTRL-GRPO | 77.62 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Non-Reasoning | 75.9 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Non-Reasoning | 75.52 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-32B Instruct | 75.43 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Non-Reasoning FTRL-GRPO | 75.33 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-7B FTRL-GRPO | 74 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-32B Non-Reasoning | 73.24 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-14B FTRL-Reinforce++ | 73.05 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-72B Instruct | 72.19 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-14B Instruct | 71.81 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-14B FTRL-GRPO | 71.52 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-7B FTRL-Reinforce++ | 70.95 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen2.5-7B Instruct | 70.38 | Not recorded | 68fa7b401e209131cea874c8 | | GPT 3.5-Turbo | 65.71 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Reasoning FTRL-GRPO | 60.57 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Reasoning FTRL-Reinforce++ | 58.95 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-14B Reasoning | 57.33 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Reasoning FTRL-GRPO | 54.57 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Reasoning | 54 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-32B Reasoning | 53.9 | Not recorded | 68fa7b401e209131cea874c8 | | Gemini 2.5-Flash | 53.62 | Not recorded | 68fa7b401e209131cea874c8 | | Qwen3-8B Reasoning FTRL-Reinforce++ | 53.52 | Not recorded | 68fa7b401e209131cea874c8 | | Gemini 2.5-Pro | 49.05 | Not recorded | 68fa7b401e209131cea874c8 | | Claude 4.0-Sonnet | 39.72 | Not recorded | 68fa7b401e209131cea874c8 | ### GPT 4o TS: 78.76. Result ID: 69e0881e47f34ce924c943a8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Version":"4o"}. This record does not assert independent reproduction. ### Qwen3-8B Non-Reasoning FTRL-Reinforce++ TS: 78.1. Result ID: 69e0881e47f34ce924c943b4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Non-Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen3-14B Non-Reasoning FTRL-Reinforce++ TS: 77.81. Result ID: 69e0881e47f34ce924c943b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Non-Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen3-8B Non-Reasoning FTRL-GRPO TS: 77.62. Result ID: 69e0881e47f34ce924c943b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Non-Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen3-14B Non-Reasoning TS: 75.9. Result ID: 69e0881e47f34ce924c943b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Non-Reasoning"}. This record does not assert independent reproduction. ### Qwen3-8B Non-Reasoning TS: 75.52. Result ID: 69e0881e47f34ce924c943b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Non-Reasoning"}. This record does not assert independent reproduction. ### Qwen2.5-32B Instruct TS: 75.43. Result ID: 69e0881e47f34ce924c943a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-32B","Version":"Instruct"}. This record does not assert independent reproduction. ### Qwen3-14B Non-Reasoning FTRL-GRPO TS: 75.33. Result ID: 69e0881e47f34ce924c943b8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Non-Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen2.5-7B FTRL-GRPO TS: 74. Result ID: 69e0881e47f34ce924c943af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-7B","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen3-32B Non-Reasoning TS: 73.24. Result ID: 69e0881e47f34ce924c943ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-32B","Version":"Non-Reasoning"}. This record does not assert independent reproduction. ### Qwen2.5-14B FTRL-Reinforce++ TS: 73.05. Result ID: 69e0881e47f34ce924c943b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-14B","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen2.5-72B Instruct TS: 72.19. Result ID: 69e0881e47f34ce924c943aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-72B","Version":"Instruct"}. This record does not assert independent reproduction. ### Qwen2.5-14B Instruct TS: 71.81. Result ID: 69e0881e47f34ce924c943b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-14B","Version":"Instruct"}. This record does not assert independent reproduction. ### Qwen2.5-14B FTRL-GRPO TS: 71.52. Result ID: 69e0881e47f34ce924c943b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-14B","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen2.5-7B FTRL-Reinforce++ TS: 70.95. Result ID: 69e0881e47f34ce924c943ae. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-7B","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen2.5-7B Instruct TS: 70.38. Result ID: 69e0881e47f34ce924c943ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen2.5-7B","Version":"Instruct"}. This record does not assert independent reproduction. ### GPT 3.5-Turbo TS: 65.71. Result ID: 69e0881e47f34ce924c943a7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Version":"3.5-Turbo"}. This record does not assert independent reproduction. ### Qwen3-14B Reasoning FTRL-GRPO TS: 60.57. Result ID: 69e0881e47f34ce924c943be. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen3-14B Reasoning FTRL-Reinforce++ TS: 58.95. Result ID: 69e0881e47f34ce924c943bd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Qwen3-14B Reasoning TS: 57.33. Result ID: 69e0881e47f34ce924c943bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-14B","Version":"Reasoning"}. This record does not assert independent reproduction. ### Qwen3-8B Reasoning FTRL-GRPO TS: 54.57. Result ID: 69e0881e47f34ce924c943bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Reasoning","Method":"FTRL-GRPO"}. This record does not assert independent reproduction. ### Qwen3-8B Reasoning TS: 54. Result ID: 69e0881e47f34ce924c943b9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Reasoning"}. This record does not assert independent reproduction. ### Qwen3-32B Reasoning TS: 53.9. Result ID: 69e0881e47f34ce924c943ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-32B","Version":"Reasoning"}. This record does not assert independent reproduction. ### Gemini 2.5-Flash TS: 53.62. Result ID: 69e0881e47f34ce924c943a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Version":"2.5-Flash"}. This record does not assert independent reproduction. ### Qwen3-8B Reasoning FTRL-Reinforce++ TS: 53.52. Result ID: 69e0881e47f34ce924c943ba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Family":"Qwen3-8B","Version":"Reasoning","Method":"FTRL-Reinforce++"}. This record does not assert independent reproduction. ### Gemini 2.5-Pro TS: 49.05. Result ID: 69e0881e47f34ce924c943a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Version":"2.5-Pro"}. This record does not assert independent reproduction. ### Claude 4.0-Sonnet TS: 39.72. Result ID: 69e0881e47f34ce924c943a6. [Reporting paper]() Recorded evaluation configuration: {"Version":"4.0-Sonnet"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.