Tool Use on RoTBench Single-turn
84.8Tool SelectionPA-Tool
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PA-ToolBackbone=Gemini-2.5-Flash, Model Category=Large Language Models2025.10 | 84.8 | 62.9 | |
| BaseModel=Claude-Sonnet-4.52025.10 | 83.3 | 67.6 | |
| BaseModel=Gemini-2.5-Flash2025.10 | 82.9 | 56.2 | |
| PA-ToolBackbone=GPT-4.1-mini, Model Category=Large Language Models2025.10 | 82.9 | 59.1 | |
| Gemini-2.5-FlashMethod=Base, Model Category=Large Language Models2025.10 | 82.9 | 56.2 | |
| BaseModel=GPT-4.1-mini2025.10 | 79.1 | 58.1 | |
| GPT-4.1-miniMethod=Base, Model Category=Large Language Models2025.10 | 79.1 | 58.1 | |
| PA-ToolBackbone=Ministral-8B, Model Category=Small Language Models2025.10 | 75.2 | 31.4 | |
| Llama3.3-70BMethod=Base, Model Category=Large Language Models2025.10 | 75.2 | 27.6 | |
| PA-ToolBackbone=Llama3.3-70B, Model Category=Large Language Models2025.10 | 74.3 | 26.7 | |
| PA-ToolBackbone=Gemini-2.5-Flash-Lite, Model Category=Small Language Models2025.10 | 73.3 | 65.7 | |
| PA-ToolBackbone=GPT-4.1-nano, Model Category=Small Language Models2025.10 | 72.4 | 44.8 | |
| Gemini-2.5-Flash-LiteMethod=Base, Model Category=Small Language Models2025.10 | 72.4 | 63.8 | |
| GPT-4.1-nanoMethod=Base, Model Category=Small Language Models2025.10 | 71.4 | 34.3 | |
| PA-ToolBackbone=Qwen3-4B, Model Category=Reasoning Models, Mode=thinking mode2025.10 | 69.5 | 27.6 | |
| PA-ToolModel=Llama3.1-8B2025.10 | 68.6 | 18.1 | |
| MostFreqModel=Llama3.1-8B2025.10 | 66.7 | 18.1 | |
| Ministral-8BMethod=Base, Model Category=Small Language Models2025.10 | 66.7 | 28.6 | |
| Qwen3-4BMethod=Base, Model Category=Reasoning Models, Mode=thinking mode2025.10 | 65.7 | 30.5 | |
| GreedyModel=Llama3.1-8B2025.10 | 63.8 | 18.1 | |
| PA-ToolModel=Llama3.2-3B2025.10 | 62.9 | 21.9 | |
| PA-ToolBackbone=Qwen3-1.7B, Model Category=Reasoning Models, Mode=thinking mode2025.10 | 62.9 | 42.9 | |
| MostFreqModel=Llama3.2-3B2025.10 | 60 | 18.1 | |
| GreedyModel=Llama3.2-3B2025.10 | 59.1 | 20 | |
| BaseModel=Llama3.1-8B2025.10 | 58.1 | 17.1 | |
| Qwen3-1.7BMethod=Base, Model Category=Reasoning Models, Mode=thinking mode2025.10 | 58.1 | 32.4 | |
| BaseModel=Llama3.2-3B2025.10 | 56.2 | 20 | |
| PA-ToolModel=Qwen2.5-7B2025.10 | 55.2 | 21.9 | |
| GreedyModel=Qwen2.5-7B2025.10 | 50.5 | 21 | |
| MostFreqModel=Qwen2.5-7B2025.10 | 50.5 | 23.8 | |
| BaseModel=Qwen2.5-7B2025.10 | 49.5 | 20 | |
| PA-ToolModel=Qwen2.5-3B2025.10 | 18.1 | 10.5 | |
| MostFreqModel=Qwen2.5-3B2025.10 | 14.3 | 8.6 | |
| GreedyModel=Qwen2.5-3B2025.10 | 13.3 | 7.6 | |
| BaseModel=Qwen2.5-3B2025.10 | 12.4 | 7.6 |