Tool Learning under Instruction with Multiple Requests on NoisyToolBench IMR 1.0 (test)
90A1 ScoreDFSDT + AwN
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DFSDT + AwNModel=gpt-4o2024.08 | 90 | 52 | 36 | |
| DFSDT + AwNModel=gpt-3.52024.08 | 88 | 56 | 48 | |
| CoT + AwNModel=gpt-3.52024.08 | 86 | 46 | 20 | |
| DFSDT + AwNModel=deepseek-v32024.08 | 82 | 52 | 48 | |
| DFSDT + AwNModel=gpt-42024.08 | 80 | 50 | 44 | |
| CoT + AwNModel=gpt-4o2024.08 | 80 | 46 | 30 | |
| CoT + AwNModel=gpt-42024.08 | 76 | 44 | 38 | |
| DFSDT + AwNModel=gemini-1.52024.08 | 76 | 28 | 4 | |
| CoT + AwNModel=O3-mini2024.08 | 76 | 54 | 48 | |
| DFSDT + AwNModel=O3-mini2024.08 | 76 | 58 | 54 | |
| CoT + AwNModel=gemini-1.52024.08 | 74 | 24 | 8 | |
| CoTModel=gpt-42024.08 | 72 | 52 | 36 | |
| DFSDTModel=gpt-42024.08 | 70 | 28 | 26 | |
| CoT + AwNModel=deepseek-v32024.08 | 70 | 54 | 46 | |
| DFSDT + AwNModel=DeepSeek-R12024.08 | 62 | 42 | 26 | |
| DFSDTModel=gpt-3.52024.08 | 60 | 18 | 16 | |
| DFSDTModel=deepseek-v32024.08 | 60 | 20 | 18 | |
| DFSDTModel=gemini-1.52024.08 | 60 | 8 | 4 | |
| CoT + AwNModel=DeepSeek-R12024.08 | 60 | 54 | 22 | |
| CoT + AwNModel=claude-3.52024.08 | 32 | 30 | 24 | |
| DFSDT + AwNModel=claude-3.52024.08 | 32 | 30 | 18 | |
| DFSDTModel=gpt-4o2024.08 | 26 | 18 | 16 | |
| CoTModel=deepseek-v32024.08 | 24 | 28 | 24 | |
| CoTModel=gemini-1.52024.08 | 22 | 10 | 2 | |
| CoTModel=gpt-3.52024.08 | 20 | 24 | 12 | |
| CoTModel=gpt-4o2024.08 | 18 | 28 | 16 | |
| CoTModel=claude-3.52024.08 | 12 | 28 | 24 | |
| DFSDTModel=claude-3.52024.08 | 12 | 18 | 18 | |
| DFSDTModel=DeepSeek-R12024.08 | 4 | 26 | 14 | |
| CoTModel=DeepSeek-R12024.08 | 2 | 28 | 12 | |
| CoTModel=O3-mini2024.08 | 0 | 16 | 14 | |
| DFSDTModel=O3-mini2024.08 | 0 | 16 | 10 |