# Tool Use on RoTBench Single-turn Source: [Tool Use on RoTBench Single-turn]() Collection index: [llms.txt]() Data updated: 2026-04-21T09:05:14.333Z Evaluation of model performance on single-turn tool utilization capabilities using the RoTBench Single-turn dataset, focusing on Tool Selection and Parameter Identification metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Tool Use | | Dataset (including any stated split) | RoTBench Single-turn | | Metric | Tool Selection | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 35 of 35 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Tool Selection](): Higher is better; unit: PERCENT. - [Parameter Identification](): Higher is better; unit: PERCENT. ## Recorded results | Method | Tool Selection | Result date | Report paper | | --- | --- | --- | --- | | PA-Tool | 84.8 | Not recorded | 68fa7c041e209131cea8be37 | | Base | 83.3 | Not recorded | 68fa7c041e209131cea8be37 | | Base | 82.9 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 82.9 | Not recorded | 68fa7c041e209131cea8be37 | | Gemini-2.5-Flash | 82.9 | Not recorded | 68fa7c041e209131cea8be37 | | Base | 79.1 | Not recorded | 68fa7c041e209131cea8be37 | | GPT-4.1-mini | 79.1 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 75.2 | Not recorded | 68fa7c041e209131cea8be37 | | Llama3.3-70B | 75.2 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 74.3 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 73.3 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 72.4 | Not recorded | 68fa7c041e209131cea8be37 | | Gemini-2.5-Flash-Lite | 72.4 | Not recorded | 68fa7c041e209131cea8be37 | | GPT-4.1-nano | 71.4 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 69.5 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 68.6 | Not recorded | 68fa7c041e209131cea8be37 | | MostFreq | 66.7 | Not recorded | 68fa7c041e209131cea8be37 | | Ministral-8B | 66.7 | Not recorded | 68fa7c041e209131cea8be37 | | Qwen3-4B | 65.7 | Not recorded | 68fa7c041e209131cea8be37 | | Greedy | 63.8 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 62.9 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 62.9 | Not recorded | 68fa7c041e209131cea8be37 | | MostFreq | 60 | Not recorded | 68fa7c041e209131cea8be37 | | Greedy | 59.1 | Not recorded | 68fa7c041e209131cea8be37 | | Base | 58.1 | Not recorded | 68fa7c041e209131cea8be37 | | Qwen3-1.7B | 58.1 | Not recorded | 68fa7c041e209131cea8be37 | | Base | 56.2 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 55.2 | Not recorded | 68fa7c041e209131cea8be37 | | Greedy | 50.5 | Not recorded | 68fa7c041e209131cea8be37 | | MostFreq | 50.5 | Not recorded | 68fa7c041e209131cea8be37 | | Base | 49.5 | Not recorded | 68fa7c041e209131cea8be37 | | PA-Tool | 18.1 | Not recorded | 68fa7c041e209131cea8be37 | | MostFreq | 14.3 | Not recorded | 68fa7c041e209131cea8be37 | | Greedy | 13.3 | Not recorded | 68fa7c041e209131cea8be37 | | Base | 12.4 | Not recorded | 68fa7c041e209131cea8be37 | ### PA-Tool Tool Selection: 84.8. Result ID: 69e73dbf47f34ce924cda77a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Gemini-2.5-Flash","Model Category":"Large Language Models"}. This record does not assert independent reproduction. ### Base Tool Selection: 83.3. Result ID: 69e73dbf47f34ce924cda73b. [Reporting paper]() Recorded evaluation configuration: {"Model":"Claude-Sonnet-4.5"}. This record does not assert independent reproduction. ### Base Tool Selection: 82.9. Result ID: 69e73dbf47f34ce924cda73a. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemini-2.5-Flash"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 82.9. Result ID: 69e73dbf47f34ce924cda778. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1-mini","Model Category":"Large Language Models"}. This record does not assert independent reproduction. ### Gemini-2.5-Flash Tool Selection: 82.9. Result ID: 69e73dbf47f34ce924cda779. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Large Language Models"}. This record does not assert independent reproduction. ### Base Tool Selection: 79.1. Result ID: 69e73dbf47f34ce924cda739. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-4.1-mini"}. This record does not assert independent reproduction. ### GPT-4.1-mini Tool Selection: 79.1. Result ID: 69e73dbf47f34ce924cda777. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Large Language Models"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 75.2. Result ID: 69e73dbf47f34ce924cda770. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Ministral-8B","Model Category":"Small Language Models"}. This record does not assert independent reproduction. ### Llama3.3-70B Tool Selection: 75.2. Result ID: 69e73dbf47f34ce924cda775. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Large Language Models"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 74.3. Result ID: 69e73dbf47f34ce924cda776. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama3.3-70B","Model Category":"Large Language Models"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 73.3. Result ID: 69e73dbf47f34ce924cda774. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Gemini-2.5-Flash-Lite","Model Category":"Small Language Models"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 72.4. Result ID: 69e73dbf47f34ce924cda772. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT-4.1-nano","Model Category":"Small Language Models"}. This record does not assert independent reproduction. ### Gemini-2.5-Flash-Lite Tool Selection: 72.4. Result ID: 69e73dbf47f34ce924cda773. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Small Language Models"}. This record does not assert independent reproduction. ### GPT-4.1-nano Tool Selection: 71.4. Result ID: 69e73dbf47f34ce924cda771. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Small Language Models"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 69.5. Result ID: 69e73dbf47f34ce924cda77e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B","Model Category":"Reasoning Models","Mode":"thinking mode"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 68.6. Result ID: 69e73dbf47f34ce924cda74b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B"}. This record does not assert independent reproduction. ### MostFreq Tool Selection: 66.7. Result ID: 69e73dbf47f34ce924cda74a. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B"}. This record does not assert independent reproduction. ### Ministral-8B Tool Selection: 66.7. Result ID: 69e73dbf47f34ce924cda76f. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Small Language Models"}. This record does not assert independent reproduction. ### Qwen3-4B Tool Selection: 65.7. Result ID: 69e73dbf47f34ce924cda77d. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Reasoning Models","Mode":"thinking mode"}. This record does not assert independent reproduction. ### Greedy Tool Selection: 63.8. Result ID: 69e73dbf47f34ce924cda749. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 62.9. Result ID: 69e73dbf47f34ce924cda747. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama3.2-3B"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 62.9. Result ID: 69e73dbf47f34ce924cda77c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-1.7B","Model Category":"Reasoning Models","Mode":"thinking mode"}. This record does not assert independent reproduction. ### MostFreq Tool Selection: 60. Result ID: 69e73dbf47f34ce924cda746. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3.2-3B"}. This record does not assert independent reproduction. ### Greedy Tool Selection: 59.1. Result ID: 69e73dbf47f34ce924cda745. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3.2-3B"}. This record does not assert independent reproduction. ### Base Tool Selection: 58.1. Result ID: 69e73dbf47f34ce924cda748. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3.1-8B"}. This record does not assert independent reproduction. ### Qwen3-1.7B Tool Selection: 58.1. Result ID: 69e73dbf47f34ce924cda77b. [Reporting paper]() Recorded evaluation configuration: {"Method":"Base","Model Category":"Reasoning Models","Mode":"thinking mode"}. This record does not assert independent reproduction. ### Base Tool Selection: 56.2. Result ID: 69e73dbf47f34ce924cda744. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama3.2-3B"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 55.2. Result ID: 69e73dbf47f34ce924cda743. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### Greedy Tool Selection: 50.5. Result ID: 69e73dbf47f34ce924cda741. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### MostFreq Tool Selection: 50.5. Result ID: 69e73dbf47f34ce924cda742. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### Base Tool Selection: 49.5. Result ID: 69e73dbf47f34ce924cda740. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B"}. This record does not assert independent reproduction. ### PA-Tool Tool Selection: 18.1. Result ID: 69e73dbf47f34ce924cda73f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### MostFreq Tool Selection: 14.3. Result ID: 69e73dbf47f34ce924cda73e. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Greedy Tool Selection: 13.3. Result ID: 69e73dbf47f34ce924cda73d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Base Tool Selection: 12.4. Result ID: 69e73dbf47f34ce924cda73c. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.