Tool-use on BFCL v3 (test)
31Base ScorePRS
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PRSModel=Qwen2.5-1.5B2026.06 | 31 | 25 | 26 | 21 | 25.75 | |
| ETSModel=Qwen2.5-1.5B2026.06 | 26 | 25 | 16 | 13 | 20 | |
| ETSModel=Qwen3-1.7B2026.06 | 26 | 27 | 19 | 21 | 23.25 | |
| SFTToolACE + RLModel=Qwen2.5-1.5B2026.06 | 23 | 23 | 13 | 10 | 17.25 | |
| SFTBFCLModel=Qwen3-1.7B2026.06 | 23 | 20 | 23 | 14 | 20 | |
| PRSModel=Qwen3-1.7B2026.06 | 23 | 22 | 17 | 16 | 19.5 | |
| SFTBFCL + RLModel=Qwen2.5-1.5B2026.06 | 21 | 22 | 19 | 7 | 17.25 | |
| SFTBFCLModel=Qwen2.5-1.5B2026.06 | 19 | 19 | 17 | 12 | 16.75 | |
| VanillaModel=Qwen3-1.7B2026.06 | 14 | 11 | 14 | 11 | 12.5 | |
| SFTToolACEModel=Qwen3-1.7B2026.06 | 12 | 10 | 5 | 8 | 8.75 | |
| VanillaModel=Qwen2.5-1.5B2026.06 | 4 | 5 | 1 | 4 | 3.5 | |
| GRPOModel=Qwen3-1.7B2026.06 | 2 | 0 | 2 | 2 | 1.5 | |
| OPSModel=Qwen2.5-1.5B2026.06 | 1 | 3 | 1 | 1 | 1.5 | |
| HBGModel=Qwen3-1.7B2026.06 | 1 | 0 | 1 | 1 | 0.75 | |
| GRPOModel=Qwen2.5-1.5B2026.06 | 0 | 0 | 0 | 0 | 0 | |
| HBGModel=Qwen2.5-1.5B2026.06 | 0 | 0 | 0 | 0 | 0 | |
| SFTBFCL + RLModel=Qwen3-1.7B2026.06 | 0 | 0 | 0 | 0 | 0 | |
| SFTToolACE + RLModel=Qwen3-1.7B2026.06 | 0 | 0 | 0 | 0 | 0 | |
| OPSModel=Qwen3-1.7B2026.06 | 0 | 0 | 0 | 0 | 0 |