Robustness of Tool-use on RoTBench
78.76TSGPT 4o
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT 4oVersion=4o2025.08 | 78.76 | 49.43 | 31.52 | |
| Qwen3-8B Non-Reasoning FTRL-Reinforce++Family=Qwen3-8B, Version=Non-Reasoning, Method=FTRL-Reinforce++2025.08 | 78.1 | 42.57 | 25.9 | |
| Qwen3-14B Non-Reasoning FTRL-Reinforce++Family=Qwen3-14B, Version=Non-Reasoning, Method=FTRL-Reinforce++2025.08 | 77.81 | 42.57 | 25.24 | |
| Qwen3-8B Non-Reasoning FTRL-GRPOFamily=Qwen3-8B, Version=Non-Reasoning, Method=FTRL-GRPO2025.08 | 77.62 | 44.67 | 27.05 | |
| Qwen3-14B Non-ReasoningFamily=Qwen3-14B, Version=Non-Reasoning2025.08 | 75.9 | 36.38 | 20.67 | |
| Qwen3-8B Non-ReasoningFamily=Qwen3-8B, Version=Non-Reasoning2025.08 | 75.52 | 36.29 | 22.19 | |
| Qwen2.5-32B InstructFamily=Qwen2.5-32B, Version=Instruct2025.08 | 75.43 | 36.86 | 20 | |
| Qwen3-14B Non-Reasoning FTRL-GRPOFamily=Qwen3-14B, Version=Non-Reasoning, Method=FTRL-GRPO2025.08 | 75.33 | 40.95 | 22.48 | |
| Qwen2.5-7B FTRL-GRPOFamily=Qwen2.5-7B, Method=FTRL-GRPO2025.08 | 74 | 30.86 | 18.95 | |
| Qwen3-32B Non-ReasoningFamily=Qwen3-32B, Version=Non-Reasoning2025.08 | 73.24 | 41.71 | 24.86 | |
| Qwen2.5-14B FTRL-Reinforce++Family=Qwen2.5-14B, Method=FTRL-Reinforce++2025.08 | 73.05 | 45.24 | 28.19 | |
| Qwen2.5-72B InstructFamily=Qwen2.5-72B, Version=Instruct2025.08 | 72.19 | 26.29 | 17.14 | |
| Qwen2.5-14B InstructFamily=Qwen2.5-14B, Version=Instruct2025.08 | 71.81 | 38.29 | 21.33 | |
| Qwen2.5-14B FTRL-GRPOFamily=Qwen2.5-14B, Method=FTRL-GRPO2025.08 | 71.52 | 36.19 | 21.62 | |
| Qwen2.5-7B FTRL-Reinforce++Family=Qwen2.5-7B, Method=FTRL-Reinforce++2025.08 | 70.95 | 40.95 | 26.1 | |
| Qwen2.5-7B InstructFamily=Qwen2.5-7B, Version=Instruct2025.08 | 70.38 | 27.33 | 15.43 | |
| GPT 3.5-TurboVersion=3.5-Turbo2025.08 | 65.71 | 29.81 | 17.33 | |
| Qwen3-14B Reasoning FTRL-GRPOFamily=Qwen3-14B, Version=Reasoning, Method=FTRL-GRPO2025.08 | 60.57 | 38.1 | 21.43 | |
| Qwen3-14B Reasoning FTRL-Reinforce++Family=Qwen3-14B, Version=Reasoning, Method=FTRL-Reinforce++2025.08 | 58.95 | 37.24 | 21.62 | |
| Qwen3-14B ReasoningFamily=Qwen3-14B, Version=Reasoning2025.08 | 57.33 | 36.19 | 20.29 | |
| Qwen3-8B Reasoning FTRL-GRPOFamily=Qwen3-8B, Version=Reasoning, Method=FTRL-GRPO2025.08 | 54.57 | 39.24 | 22.95 | |
| Qwen3-8B ReasoningFamily=Qwen3-8B, Version=Reasoning2025.08 | 54 | 34.29 | 20.38 | |
| Qwen3-32B ReasoningFamily=Qwen3-32B, Version=Reasoning2025.08 | 53.9 | 33.81 | 20.19 | |
| Gemini 2.5-FlashVersion=2.5-Flash2025.08 | 53.62 | 34.95 | 22.76 | |
| Qwen3-8B Reasoning FTRL-Reinforce++Family=Qwen3-8B, Version=Reasoning, Method=FTRL-Reinforce++2025.08 | 53.52 | 37.52 | 22.29 | |
| Gemini 2.5-ProVersion=2.5-Pro2025.08 | 49.05 | 42.38 | 27.52 | |
| Claude 4.0-SonnetVersion=4.0-Sonnet2025.08 | 39.72 | 34.95 | 21.81 |