Multi-turn Tool-use on BFCL
27.5Multi-Turn Success RatePACT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PACTBackbone=Qwen3-4B2026.06 | 27.5 | 18 | 18.28 | 21.26 | |
| MatchTIRBackbone=Qwen3-4B2026.06 | 24.38 | 17.5 | 14.62 | 18.83 | |
| SFT→MatchTIRBackbone=Qwen3-4B2026.06 | 23.62 | 17.85 | 13.76 | 18.41 | |
| VanillaBackbone=Qwen3-4B2026.06 | 23.5 | 12 | 16.34 | 17.28 | |
| ToolRLBackbone=Qwen3-4B2026.06 | 23 | 11 | 17.42 | 17.14 | |
| SFTBackbone=Qwen3-4B2026.06 | 22.62 | 4 | 12.9 | 13.17 | |
| FTRLBackbone=Qwen3-4B2026.06 | 22.38 | 9 | 16.77 | 16.05 | |
| GRPOBackbone=Qwen3-4B2026.06 | 22 | 9 | 15.48 | 15.49 | |
| CHORDBackbone=Qwen3-4B2026.06 | 22 | 12.5 | 15.48 | 16.66 | |
| PACTBackbone=Qwen3-1.7B2026.06 | 14.88 | 7 | 13.63 | 11.84 | |
| MatchTIRBackbone=Qwen3-1.7B2026.06 | 14 | 4.5 | 9.03 | 9.18 | |
| SFT→MatchTIRBackbone=Qwen3-1.7B2026.06 | 13.63 | 5.5 | 11.18 | 10.1 | |
| SFTBackbone=Qwen3-1.7B2026.06 | 13.38 | 2.5 | 8.6 | 8.16 | |
| ToolRLBackbone=Qwen3-1.7B2026.06 | 13.3 | 2.5 | 8.39 | 8.06 | |
| VanillaBackbone=Qwen3-1.7B2026.06 | 13 | 1 | 8.6 | 7.53 | |
| FTRLBackbone=Qwen3-1.7B2026.06 | 12.62 | 6 | 11.18 | 9.93 | |
| CHORDBackbone=Qwen3-1.7B2026.06 | 12.62 | 3.5 | 6.45 | 7.52 | |
| GRPOBackbone=Qwen3-1.7B2026.06 | 11.88 | 3.5 | 9.25 | 8.21 |