Tool-using on NoisyToolBench IMKI
1.26Average StepsDFSDT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DFSDTModel=DeepSeek-R12024.08 | 1.26 | — | |
| CoTModel=DeepSeek-R12024.08 | 1.3 | — | |
| CoTModel=O3-mini2024.08 | 1.88 | — | |
| DFSDTModel=O3-mini2024.08 | 2.04 | — | |
| AwNModel=DeepSeek-R1, Framework=CoT2024.08 | 2.5 | 0.16 | |
| AwNModel=DeepSeek-R1, Framework=DFSDT2024.08 | 2.54 | 0.22 | |
| CoTModel=claude-3.52024.08 | 2.64 | — | |
| CoTModel=gpt-4o2024.08 | 3 | — | |
| DFSDTModel=claude-3.52024.08 | 3.34 | — | |
| AwNModel=O3-mini, Framework=CoT2024.08 | 3.54 | 0.42 | |
| AwNModel=O3-mini, Framework=DFSDT2024.08 | 3.62 | 0.44 | |
| AwNModel=claude-3.5, Framework=CoT2024.08 | 3.74 | 0.18 | |
| AwNModel=gpt-4o, Framework=CoT2024.08 | 3.86 | 0.62 | |
| AwNModel=deepseek-v3, Framework=CoT2024.08 | 3.88 | 0.2 | |
| AwNModel=gpt-4, Framework=CoT2024.08 | 3.94 | 0.16 | |
| CoTModel=gpt-42024.08 | 4 | — | |
| CoTModel=gemini-1.52024.08 | 4 | — | |
| CoTModel=deepseek-v32024.08 | 4.2 | — | |
| CoTModel=gpt-3.52024.08 | 4.46 | — | |
| AwNModel=gpt-3.5, Framework=CoT2024.08 | 5.36 | 0.66 | |
| DFSDTModel=gpt-4o2024.08 | 5.98 | — | |
| AwNModel=gemini-1.5, Framework=CoT2024.08 | 6.44 | 0.42 | |
| AwNModel=gpt-4o, Framework=DFSDT2024.08 | 6.7 | 0.86 | |
| AwNModel=claude-3.5, Framework=DFSDT2024.08 | 6.74 | 0.76 | |
| AwNModel=gpt-4, Framework=DFSDT2024.08 | 9.82 | 0.48 | |
| DFSDTModel=gpt-3.52024.08 | 12.82 | — | |
| AwNModel=deepseek-v3, Framework=DFSDT2024.08 | 15.86 | 1.16 | |
| AwNModel=gpt-3.5, Framework=DFSDT2024.08 | 16.94 | 1.44 | |
| DFSDTModel=deepseek-v32024.08 | 59.08 | — | |
| DFSDTModel=gpt-42024.08 | 83.96 | — | |
| AwNModel=gemini-1.5, Framework=DFSDT2024.08 | 445.16 | 5.34 | |
| DFSDTModel=gemini-1.52024.08 | 750.8 | — |