Tool-using on NoisyToolBench IBTC
1Average StepsCoT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CoTModel=DeepSeek-R12024.08 | 1 | — | |
| AwNModel=DeepSeek-R1, Framework=CoT2024.08 | 1.02 | 0.02 | |
| AwNModel=DeepSeek-R1, Framework=DFSDT2024.08 | 1.02 | 0 | |
| DFSDTModel=O3-mini2024.08 | 1.04 | — | |
| CoTModel=O3-mini2024.08 | 1.08 | — | |
| AwNModel=gpt-4o, Framework=CoT2024.08 | 1.1 | 0 | |
| AwNModel=deepseek-v3, Framework=CoT2024.08 | 1.1 | 0.02 | |
| AwNModel=gpt-4o, Framework=DFSDT2024.08 | 1.14 | 0 | |
| AwNModel=gpt-4, Framework=CoT2024.08 | 1.16 | 0.04 | |
| DFSDTModel=DeepSeek-R12024.08 | 1.16 | — | |
| CoTModel=deepseek-v32024.08 | 1.18 | — | |
| CoTModel=gpt-4o2024.08 | 1.28 | — | |
| AwNModel=deepseek-v3, Framework=DFSDT2024.08 | 1.32 | 0.04 | |
| AwNModel=gemini-1.5, Framework=CoT2024.08 | 1.46 | 0.46 | |
| AwNModel=gemini-1.5, Framework=DFSDT2024.08 | 1.46 | 0.46 | |
| AwNModel=O3-mini, Framework=CoT2024.08 | 1.54 | 0.3 | |
| AwNModel=O3-mini, Framework=DFSDT2024.08 | 1.54 | 0.24 | |
| AwNModel=claude-3.5, Framework=CoT2024.08 | 1.68 | 0.1 | |
| CoTModel=claude-3.52024.08 | 1.9 | — | |
| CoTModel=gpt-42024.08 | 2.04 | — | |
| CoTModel=gpt-3.52024.08 | 2.1 | — | |
| AwNModel=gpt-4, Framework=DFSDT2024.08 | 2.1 | 0.16 | |
| AwNModel=gpt-3.5, Framework=CoT2024.08 | 2.4 | 1.1 | |
| AwNModel=claude-3.5, Framework=DFSDT2024.08 | 2.76 | 0.14 | |
| AwNModel=gpt-3.5, Framework=DFSDT2024.08 | 3.94 | 1.7 | |
| DFSDTModel=gpt-42024.08 | 4.06 | — | |
| DFSDTModel=claude-3.52024.08 | 4.26 | — | |
| CoTModel=gemini-1.52024.08 | 4.52 | — | |
| DFSDTModel=gpt-3.52024.08 | 5.5 | — | |
| DFSDTModel=gpt-4o2024.08 | 8.98 | — | |
| DFSDTModel=deepseek-v32024.08 | 11.64 | — | |
| DFSDTModel=gemini-1.52024.08 | 559.78 | — |