Tool Use on RobustBench-TC Clean 1.0 (test)
77.9Clean AccuracyLoopTool-32B
Evaluation Results
| Method | Links | |
|---|---|---|
| LoopTool-32BModel Category=Open-source RL tool models, Model Scale=32B2026.05 | 77.9 | |
| Qwen3-32BModel Category=Base / instruct counterparts, Model Scale=32B2026.05 | 75.4 | |
| Qwen3-14BModel Category=Base / instruct counterparts, Model Scale=14B2026.05 | 74.9 | |
| LoopTool-8BModel Category=Open-source RL tool models, Model Scale=8B2026.05 | 71.4 | |
| Qwen3.5-9BModel Category=Frontier, Model Scale=9B2026.05 | 70.9 | |
| o4-mini (OpenAI)Model Category=Closed-source frontier2026.05 | 70.9 | |
| ToolRL-Qwen2.5-1.5BModel Category=Open-source RL tool models, Model Scale=1.5B2026.05 | 70.4 | |
| MUA-RL-32BModel Category=Open-source RL tool models, Model Scale=32B2026.05 | 68.3 | |
| MUA-RL-8BModel Category=Open-source RL tool models, Model Scale=8B2026.05 | 67.8 | |
| Qwen3-8BModel Category=Base / instruct counterparts, Model Scale=8B2026.05 | 67.3 | |
| ToolRL-DR-Mixed (ours)Model Category=Our 3B trained checkpoints, Model Scale=3B2026.05 | 65.8 | |
| TL-CodeLLaMA-2Model Category=Open-source RL tool models, Model Scale=7B2026.05 | 65.3 | |
| ToolRL-DR-Full (ours)Model Category=Our 3B trained checkpoints, Model Scale=3B2026.05 | 64.3 | |
| ToolRL-Qwen2.5-3BModel Category=Open-source RL tool models, Model Scale=3B2026.05 | 63.8 | |
| MUA-RL-14BModel Category=Open-source RL tool models, Model Scale=14B2026.05 | 62.8 | |
| ToolRL-Llama3.2-3BModel Category=Open-source RL tool models, Model Scale=3B2026.05 | 62.8 | |
| DeepSeek-R1-Distill-14BModel Category=Frontier, Model Scale=14B2026.05 | 61.8 | |
| Qwen2.5-3B-InstructModel Category=Base / instruct counterparts, Model Scale=3B2026.05 | 60.8 | |
| SFT-Clean-4k-3BModel Category=ToolRL SFT version, Model Scale=3B2026.05 | 59.8 | |
| Qwen2.5-1.5B-InstructModel Category=Base / instruct counterparts, Model Scale=1.5B2026.05 | 57.3 | |
| Llama-3.2-3B-InstructModel Category=Base / instruct counterparts, Model Scale=3B2026.05 | 51.8 |