Tool-using on NoisyToolBench IMR
1.03Average StepsAwN
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| AwNModel=claude-3.5, Framework=CoT2024.08 | 1.03 | 0.33 | |
| CoTModel=DeepSeek-R12024.08 | 1.22 | — | |
| AwNModel=DeepSeek-R1, Framework=CoT2024.08 | 1.94 | 0.2 | |
| DFSDTModel=DeepSeek-R12024.08 | 2.16 | — | |
| CoTModel=gpt-4o2024.08 | 2.98 | — | |
| CoTModel=O3-mini2024.08 | 3.08 | — | |
| CoTModel=claude-3.52024.08 | 3.4 | — | |
| CoTModel=deepseek-v32024.08 | 3.52 | — | |
| AwNModel=deepseek-v3, Framework=CoT2024.08 | 3.6 | 0.06 | |
| AwNModel=O3-mini, Framework=CoT2024.08 | 3.82 | 0.52 | |
| AwNModel=gpt-4, Framework=CoT2024.08 | 3.94 | 0.2 | |
| AwNModel=gpt-4o, Framework=CoT2024.08 | 3.96 | 0.7 | |
| CoTModel=gpt-42024.08 | 3.98 | — | |
| CoTModel=gpt-3.52024.08 | 4.02 | — | |
| AwNModel=DeepSeek-R1, Framework=DFSDT2024.08 | 4.08 | 0.36 | |
| AwNModel=O3-mini, Framework=DFSDT2024.08 | 4.1 | 0.4 | |
| CoTModel=gemini-1.52024.08 | 4.12 | — | |
| DFSDTModel=O3-mini2024.08 | 4.36 | — | |
| AwNModel=gpt-3.5, Framework=CoT2024.08 | 5.98 | 1.1 | |
| AwNModel=gemini-1.5, Framework=CoT2024.08 | 6.36 | 0.68 | |
| AwNModel=gpt-4o, Framework=DFSDT2024.08 | 7.68 | 1.18 | |
| DFSDTModel=gpt-4o2024.08 | 9.58 | — | |
| DFSDTModel=claude-3.52024.08 | 9.64 | — | |
| AwNModel=gpt-3.5, Framework=DFSDT2024.08 | 11.24 | 0.98 | |
| DFSDTModel=gpt-3.52024.08 | 12.8 | — | |
| AwNModel=gpt-4, Framework=DFSDT2024.08 | 13.08 | 0.74 | |
| AwNModel=claude-3.5, Framework=DFSDT2024.08 | 17.46 | 0.8 | |
| DFSDTModel=gpt-42024.08 | 21.04 | — | |
| AwNModel=deepseek-v3, Framework=DFSDT2024.08 | 24.6 | 1.8 | |
| DFSDTModel=deepseek-v32024.08 | 41.7 | — | |
| AwNModel=gemini-1.5, Framework=DFSDT2024.08 | 532.56 | 9.08 | |
| DFSDTModel=gemini-1.52024.08 | 685 | — |