Function Calling on ToolBench I3-Inst
52.4Pass RateClaude-3.5-Sonnet + TAFC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Claude-3.5-Sonnet + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 52.4 | 82.3 | |
| GPT-4o + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 51.2 | 81.5 | |
| Claude-3.5-SonnetFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 50.2 | 80.5 | |
| GPT-4oFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 49.1 | 79.7 | |
| Qwen2.5-72B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 40.5 | 50.2 | |
| Llama-3.1-70B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 38.3 | 47.7 | |
| Qwen2.5-72BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 38.3 | 47.7 | |
| Llama-3.1-70BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 35.8 | 44.9 | |
| Qwen2.5-32B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 34.7 | 43.8 | |
| Qwen2.5-32BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 32.4 | 41 | |
| Qwen2.5-7B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 24.1 | 30.5 | |
| Llama-3.1-8B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 22.5 | 28.7 | |
| Qwen2.5-7BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 21.7 | 27.3 | |
| Llama-3.1-8BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 20.1 | 25.6 |