Tool Use on RobustBench-TC Perturbed 1.0 (test)
52.9Accuracy (Perturbed)Qwen3-14B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Qwen3-14BModel Category=Base / instruct counterparts, Model Scale=14B2026.05 | 52.9 | 0.039 | 0.143 | 0.464 | 0.25 | |
| MUA-RL-32BModel Category=Open-source RL tool models, Model Scale=32B2026.05 | 52.7 | 0.028 | -0.01 | 0.397 | 0.218 | |
| Qwen3-32BModel Category=Base / instruct counterparts, Model Scale=32B2026.05 | 52.4 | 0.024 | 0.104 | 0.366 | 0.377 | |
| LoopTool-32BModel Category=Open-source RL tool models, Model Scale=32B2026.05 | 50.8 | 0.034 | 0.12 | 0.509 | 0.396 | |
| o4-mini (OpenAI)Model Category=Closed-source frontier2026.05 | 50.2 | 0.049 | 0.157 | 0.326 | 0.276 | |
| MUA-RL-8BModel Category=Open-source RL tool models, Model Scale=8B2026.05 | 47.9 | 0.023 | 0.156 | 0.401 | 0.229 | |
| LoopTool-8BModel Category=Open-source RL tool models, Model Scale=8B2026.05 | 47.6 | 0.049 | 0.173 | 0.429 | 0.296 | |
| MUA-RL-14BModel Category=Open-source RL tool models, Model Scale=14B2026.05 | 47 | 0.011 | 0.09 | 0.334 | 0.202 | |
| ToolRL-DR-Full (ours)Model Category=Our 3B trained checkpoints, Model Scale=3B2026.05 | 46.3 | 0.009 | 0.147 | 0.331 | 0.231 | |
| ToolRL-DR-Mixed (ours)Model Category=Our 3B trained checkpoints, Model Scale=3B2026.05 | 46.1 | 0.049 | 0.152 | 0.364 | 0.232 | |
| Qwen3-8BModel Category=Base / instruct counterparts, Model Scale=8B2026.05 | 44.8 | 0.019 | 0.171 | 0.409 | 0.293 | |
| DeepSeek-R1-Distill-14BModel Category=Frontier, Model Scale=14B2026.05 | 44.2 | 0.013 | 0.16 | 0.318 | 0.213 | |
| Qwen3.5-9BModel Category=Frontier, Model Scale=9B2026.05 | 43.7 | 0.04 | 0.168 | 0.411 | 0.417 | |
| ToolRL-Qwen2.5-1.5BModel Category=Open-source RL tool models, Model Scale=1.5B2026.05 | 43.2 | 0.033 | 0.174 | 0.481 | 0.379 | |
| TL-CodeLLaMA-2Model Category=Open-source RL tool models, Model Scale=7B2026.05 | 40.5 | 0.024 | 0.166 | 0.456 | 0.333 | |
| ToolRL-Qwen2.5-3BModel Category=Open-source RL tool models, Model Scale=3B2026.05 | 40.4 | 0.02 | 0.143 | 0.45 | 0.315 | |
| ToolRL-Llama3.2-3BModel Category=Open-source RL tool models, Model Scale=3B2026.05 | 39 | 0.025 | 0.171 | 0.463 | 0.296 | |
| Qwen2.5-3B-InstructModel Category=Base / instruct counterparts, Model Scale=3B2026.05 | 39 | 0.014 | 0.195 | 0.446 | 0.237 | |
| Qwen2.5-1.5B-InstructModel Category=Base / instruct counterparts, Model Scale=1.5B2026.05 | 36.7 | 0.04 | 0.2 | 0.405 | 0.205 | |
| Llama-3.2-3B-InstructModel Category=Base / instruct counterparts, Model Scale=3B2026.05 | 34.1 | 0.009 | 0.177 | 0.372 | 0.174 | |
| SFT-Clean-4k-3BModel Category=ToolRL SFT version, Model Scale=3B2026.05 | 26.7 | 0.019 | 0.184 | 0.443 | 0.577 |