Tool-augmented reasoning on BFCL Multi-Turn v3
69.1Overall ScoreAPIGen-MT
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| APIGen-MTBase Model=Llama3.1-8B-Inst2026.01 | 69.1 | 77 | 69.5 | 63 | 67 | |
| GPT-4o-2024-11-202026.01 | 50 | 61 | 45.5 | 35.5 | 58 | |
| ToolACE-MT-8BBase Model=Llama3.1-8B-Inst2026.01 | 40.3 | 57.5 | 31.5 | 34 | 38 | |
| Qwen3-8B2026.01 | 37 | 42.5 | 38.5 | 31.5 | 35.5 | |
| Gemini-2.5-Pro-Preview-05-062026.01 | 34.6 | 39.5 | 29.5 | 31.5 | 38 | |
| MASBase Model=Llama3.1-8B-Inst2026.01 | 31.4 | 46.5 | 19 | 31 | 29 | |
| DeepSeek-V3-03242026.01 | 29.9 | 41 | 21 | 23 | 34.5 | |
| DiGiT-TCBase Model=Llama3.1-8B-Inst2026.01 | 27 | 42 | 16 | 17 | 33 | |
| MagnetBase Model=Qwen-Coder-7B-Inst2026.01 | 26.5 | 35.5 | 24 | 27.5 | 19 | |
| ToolWeaveBase Model=Llama3.1-8B-Inst2026.01 | 25 | 31.5 | 20.5 | 24.5 | 23.5 | |
| ToucanBase Model=Qwen2.5-7B-Inst2026.01 | 22.6 | — | — | — | — | |
| Mistral-Small-25062026.01 | 17.6 | 22.5 | 11.5 | 19 | 17.5 | |
| Llama3.1-70B-Inst2026.01 | 12.5 | 17 | 13 | 10.5 | 9.5 | |
| Llama3.1-8B-Inst2026.01 | 9.3 | 12 | 10 | 7 | 8 |