Tool-use Reasoning on Ours (test)
52.78Solve Precision (P)Gemini 2.5-Flash
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gemini 2.5-FlashVersion=2.5-Flash2025.08 | 52.78 | 23.13 | 23.24 | |
| Gemini 2.5-ProVersion=2.5-Pro2025.08 | 51.77 | 31.96 | 29.98 | |
| Qwen2.5-14B FTRL-Reinforce++Family=Qwen2.5-14B, Method=FTRL-Reinforce++2025.08 | 49.51 | 47.2 | 47.57 | |
| Qwen3-14B Non-Reasoning FTRL-GRPOFamily=Qwen3-14B, Version=Non-Reasoning, Method=FTRL-GRPO2025.08 | 49.41 | 56.71 | 51.82 | |
| Qwen2.5-14B FTRL-GRPOFamily=Qwen2.5-14B, Method=FTRL-GRPO2025.08 | 48.67 | 50.56 | 49.2 | |
| Qwen2.5-7B FTRL-GRPOFamily=Qwen2.5-7B, Method=FTRL-GRPO2025.08 | 47.82 | 47.37 | 46.78 | |
| Qwen3-8B Non-Reasoning FTRL-GRPOFamily=Qwen3-8B, Version=Non-Reasoning, Method=FTRL-GRPO2025.08 | 46.44 | 55.84 | 49.54 | |
| Qwen3-14B Reasoning FTRL-Reinforce++Family=Qwen3-14B, Version=Reasoning, Method=FTRL-Reinforce++2025.08 | 44.15 | 48.11 | 44.04 | |
| Qwen3-14B Non-Reasoning FTRL-Reinforce++Family=Qwen3-14B, Version=Non-Reasoning, Method=FTRL-Reinforce++2025.08 | 44.06 | 59.3 | 48.24 | |
| Claude 4.0-SonnetVersion=4.0-Sonnet2025.08 | 42.33 | 56.07 | 42.59 | |
| Qwen3-8B Non-Reasoning FTRL-Reinforce++Family=Qwen3-8B, Version=Non-Reasoning, Method=FTRL-Reinforce++2025.08 | 41.96 | 47.37 | 43.63 | |
| Qwen3-14B Reasoning FTRL-GRPOFamily=Qwen3-14B, Version=Reasoning, Method=FTRL-GRPO2025.08 | 41.67 | 47.67 | 43.36 | |
| Qwen2.5-7B FTRL-Reinforce++Family=Qwen2.5-7B, Method=FTRL-Reinforce++2025.08 | 41.61 | 41.36 | 40.36 | |
| Qwen3-8B Reasoning FTRL-GRPOFamily=Qwen3-8B, Version=Reasoning, Method=FTRL-GRPO2025.08 | 40.67 | 40.06 | 38.58 | |
| Qwen3-8B Reasoning FTRL-Reinforce++Family=Qwen3-8B, Version=Reasoning, Method=FTRL-Reinforce++2025.08 | 38.11 | 43.7 | 39.4 | |
| GPT 4oVersion=4o2025.08 | 33.43 | 42.01 | 33.39 | |
| Qwen2.5-14B InstructFamily=Qwen2.5-14B, Version=Instruct2025.08 | 33.31 | 34.64 | 32.76 | |
| Qwen3-32B Non-ReasoningFamily=Qwen3-32B, Version=Non-Reasoning2025.08 | 33.2 | 46.24 | 36.34 | |
| Qwen3-14B ReasoningFamily=Qwen3-14B, Version=Reasoning2025.08 | 31.91 | 40.23 | 32.96 | |
| Qwen2.5-72B InstructFamily=Qwen2.5-72B, Version=Instruct2025.08 | 31.68 | 43.41 | 35.22 | |
| Qwen2.5-32B InstructFamily=Qwen2.5-32B, Version=Instruct2025.08 | 30.12 | 37.8 | 32.42 | |
| Qwen3-32B ReasoningFamily=Qwen3-32B, Version=Reasoning2025.08 | 28.94 | 40.41 | 31.3 | |
| Qwen3-14B Non-ReasoningFamily=Qwen3-14B, Version=Non-Reasoning2025.08 | 28.29 | 37.58 | 29.97 | |
| Qwen3-8B ReasoningFamily=Qwen3-8B, Version=Reasoning2025.08 | 27.78 | 38.38 | 29.52 | |
| Qwen2.5-7B InstructFamily=Qwen2.5-7B, Version=Instruct2025.08 | 27.44 | 27.69 | 25.97 | |
| GPT 3.5-TurboVersion=3.5-Turbo2025.08 | 25.77 | 29.29 | 25.66 | |
| Qwen3-8B Non-ReasoningFamily=Qwen3-8B, Version=Non-Reasoning2025.08 | 21.18 | 30.71 | 23.48 |