Agentic Tool-use on ACEBench (agent-task)
85.8Average Success RateDFSDT + EvoSOP
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DFSDT + EvoSOPBackbone=Qwen-Max2026.07 | 85.8 | — | — | — | |
| DFSDT + EvoSOPBackbone=GPT-4o2026.07 | 85 | — | — | — | |
| ReAct + EvoSOPBackbone=GPT-4o2026.07 | 84.2 | — | — | — | |
| ReAct + EvoSOPBackbone=Qwen-Max2026.07 | 84.2 | — | — | — | |
| DFSDT + DRAFTBackbone=Gemini-3-FP2026.07 | 84.2 | — | — | — | |
| ReAct + EvoSOPBackbone=Gemini-3-FP2026.07 | 83.3 | — | — | — | |
| DFSDT + EvoSOPBackbone=Gemini-3-FP2026.07 | 82.5 | — | — | — | |
| ReActBackbone=GPT-4o2026.07 | 80.8 | — | — | — | |
| DFSDTBackbone=GPT-4o2026.07 | 78.3 | — | — | — | |
| DFSDT + DRAFTBackbone=GPT-4o2026.07 | 75 | — | — | — | |
| DFSDT + DRAFTBackbone=Qwen-Max2026.07 | 74.2 | — | — | — | |
| ReAct + ASIBackbone=GPT-4o2026.07 | 60 | — | — | — | |
| ReAct + ASIBackbone=Qwen-Max2026.07 | 53.3 | — | — | — | |
| ReAct + ASIBackbone=Gemini-3-FP2026.07 | 45.8 | — | — | — | |
| Llama-3.1-8B-Instruct + RODS (ours)Training=RODS2026.06 | 11.15 | 14.3 | 8 | — | |
| ToolACE-2-8B2026.06 | 11 | 15 | 7 | — | |
| xLAM-2-8b-fc-r2026.06 | 9.15 | 13.3 | 5 | — | |
| Llama-3.1-8B-Instruct + EnvTuningTraining=EnvTuning2026.06 | 8.82 | 11.3 | 6.33 | — | |
| BitAgent-8B2026.06 | 8.65 | 12 | 5.3 | — | |
| Llama-3.1-8B-Instruct2026.06 | 4.15 | 3.3 | 5 | — | |
| Astra-14B-thinking-v1Temperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 54.17 | 83.75 | 68.96 | |
| Astra-32B-thinking-v1Temperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 60 | 83.75 | 71.88 | |
| Claude-Haiku-4.5Temperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 64.17 | 88.75 | 76.46 | |
| Claude-Opus-4.5Temperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 64.17 | 100 | 82.09 | |
| Claude-Sonnet-4.5Temperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 65.83 | 94.38 | 80.11 | |
| Gemini-2.5-ProTemperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 97.5 | 40 | 68.75 | |
| Gemini-3-ProTemperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 52.09 | 100 | 76.05 | |
| GLM-4.6Temperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 72.5 | 87.5 | 80 | |
| GPT-4.1Temperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 66.67 | 95 | 80.84 | |
| Kimi-K2-InstructTemperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 69.17 | 92.5 | 80.84 | |
| LoopTool-32BTemperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 57.5 | 60 | 58.75 | |
| Qwen3-14BTemperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 45.83 | 57.5 | 51.67 | |
| Qwen3-32BTemperature=0.6, User Simulator=GPT-4.1, Repetitions=42026.01 | — | 55.83 | 63.75 | 59.79 |