Tool-using on NoisyToolBench IwE
1.3Average StepsCoT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CoTModel=DeepSeek-R12024.08 | 1.3 | — | |
| AwNModel=DeepSeek-R1, Framework=CoT2024.08 | 2 | 0.12 | |
| DFSDTModel=DeepSeek-R12024.08 | 2.16 | — | |
| CoTModel=gpt-4o2024.08 | 2.48 | — | |
| CoTModel=O3-mini2024.08 | 2.72 | — | |
| CoTModel=gemini-1.52024.08 | 2.86 | — | |
| DFSDTModel=O3-mini2024.08 | 2.88 | — | |
| AwNModel=deepseek-v3, Framework=CoT2024.08 | 2.92 | 0.04 | |
| CoTModel=claude-3.52024.08 | 3.04 | — | |
| CoTModel=deepseek-v32024.08 | 3.12 | — | |
| AwNModel=gpt-4o, Framework=CoT2024.08 | 3.18 | 0.46 | |
| CoTModel=gpt-42024.08 | 3.34 | — | |
| AwNModel=gpt-4, Framework=CoT2024.08 | 3.46 | 0.36 | |
| AwNModel=O3-mini, Framework=CoT2024.08 | 3.66 | 0.44 | |
| AwNModel=DeepSeek-R1, Framework=DFSDT2024.08 | 3.66 | 0.4 | |
| AwNModel=claude-3.5, Framework=CoT2024.08 | 3.76 | 0.36 | |
| CoTModel=gpt-3.52024.08 | 3.9 | — | |
| AwNModel=O3-mini, Framework=DFSDT2024.08 | 4.16 | 0.48 | |
| AwNModel=gemini-1.5, Framework=CoT2024.08 | 4.54 | 0.48 | |
| AwNModel=gpt-3.5, Framework=CoT2024.08 | 5.08 | 0.76 | |
| DFSDTModel=gpt-4o2024.08 | 5.78 | — | |
| DFSDTModel=claude-3.52024.08 | 5.98 | — | |
| AwNModel=gpt-4o, Framework=DFSDT2024.08 | 8.56 | 0.88 | |
| AwNModel=gpt-4, Framework=DFSDT2024.08 | 9.42 | 0.62 | |
| AwNModel=gpt-3.5, Framework=DFSDT2024.08 | 11.68 | 0.94 | |
| AwNModel=deepseek-v3, Framework=DFSDT2024.08 | 11.82 | 1.2 | |
| AwNModel=claude-3.5, Framework=DFSDT2024.08 | 13.08 | 1.04 | |
| DFSDTModel=gpt-3.52024.08 | 13.82 | — | |
| DFSDTModel=gpt-42024.08 | 22.4 | — | |
| DFSDTModel=deepseek-v32024.08 | 24.24 | — | |
| AwNModel=gemini-1.5, Framework=DFSDT2024.08 | 411.18 | 1.94 | |
| DFSDTModel=gemini-1.52024.08 | 725.14 | — |