Function Calling on ToolBench I2-Inst
71.4Pass RateClaude-3.5-Sonnet + TAFC
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Claude-3.5-Sonnet + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 71.4 | 70.1 | |
| GPT-4o + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 70.2 | 69.1 | |
| Claude-3.5-SonnetFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 69.7 | 68.2 | |
| GPT-4oFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 68.5 | 67.3 | |
| Qwen2.5-72B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 57.8 | 64.9 | |
| Qwen2.5-72BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 55.4 | 62.9 | |
| Llama-3.1-70B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 54.5 | 62 | |
| Llama-3.1-70BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 52.2 | 59.6 | |
| Qwen2.5-32B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 50.3 | 57.9 | |
| Qwen2.5-32BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 47.9 | 55.3 | |
| Qwen2.5-7B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 37.5 | 44 | |
| Llama-3.1-8B + TAFCFramework=TAFC, Temperature=0.1, Independent Runs=32026.01 | 35.9 | 42.2 | |
| Qwen2.5-7BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 34.8 | 41.1 | |
| Llama-3.1-8BFramework=Standard FC, Temperature=0.1, Independent Runs=32026.01 | 33.3 | 39.4 |