Tool Learning under Instructions Beyond Tool Capabilities on NoisyToolBench IBTC 1.0 (test)
98A1 ScoreCoT + AwN
Evaluation Results
| Method | Links | |
|---|---|---|
| CoT + AwNModel=deepseek-v32024.08 | 98 | |
| DFSDT + AwNModel=deepseek-v32024.08 | 98 | |
| CoT + AwNModel=gpt-42024.08 | 94 | |
| DFSDT + AwNModel=gpt-42024.08 | 94 | |
| DFSDT + AwNModel=gpt-4o2024.08 | 94 | |
| CoT + AwNModel=gpt-4o2024.08 | 92 | |
| CoT + AwNModel=claude-3.52024.08 | 88 | |
| DFSDT + AwNModel=claude-3.52024.08 | 86 | |
| CoT + AwNModel=DeepSeek-R12024.08 | 84 | |
| DFSDT + AwNModel=DeepSeek-R12024.08 | 76 | |
| DFSDT + AwNModel=gemini-1.52024.08 | 70 | |
| CoT + AwNModel=gemini-1.52024.08 | 68 | |
| DFSDT + AwNModel=gpt-3.52024.08 | 64 | |
| DFSDTModel=claude-3.52024.08 | 62 | |
| DFSDTModel=gpt-42024.08 | 54 | |
| CoTModel=gemini-1.52024.08 | 52 | |
| CoT + AwNModel=gpt-3.52024.08 | 48 | |
| DFSDTModel=deepseek-v32024.08 | 48 | |
| DFSDTModel=gemini-1.52024.08 | 48 | |
| DFSDT + AwNModel=O3-mini2024.08 | 48 | |
| DFSDTModel=gpt-3.52024.08 | 46 | |
| CoT + AwNModel=O3-mini2024.08 | 40 | |
| CoTModel=gpt-3.52024.08 | 38 | |
| CoTModel=gpt-42024.08 | 34 | |
| CoTModel=deepseek-v32024.08 | 30 | |
| CoTModel=claude-3.52024.08 | 30 | |
| CoTModel=gpt-4o2024.08 | 10 | |
| DFSDTModel=gpt-4o2024.08 | 8 | |
| CoTModel=O3-mini2024.08 | 0 | |
| DFSDTModel=O3-mini2024.08 | 0 | |
| CoTModel=DeepSeek-R12024.08 | 0 | |
| DFSDTModel=DeepSeek-R12024.08 | 0 |