Tool-use Generalization on ACEBench Content OOD v1 (test)
40AdherenceVanilla
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| VanillaBackbone=Qwen2.5-1.5B-Instruct2026.06 | 40 | 32 | 52 | 2.5 | 31.6 | |
| SFT_BFCLBackbone=Qwen2.5-1.5B-Instruct2026.06 | 30 | 20 | 44 | 2.5 | 24.1 | |
| ETSBackbone=Qwen2.5-1.5B-Instruct2026.06 | 26 | 22 | 36 | 2.5 | 21.6 | |
| PRSBackbone=Qwen2.5-1.5B-Instruct2026.06 | 26 | 30 | 43 | 4.2 | 25.8 | |
| OPSBackbone=Qwen2.5-1.5B-Instruct2026.06 | 20 | 16 | 50 | 6.7 | 23.2 | |
| SFT_BFCL + RLBackbone=Qwen2.5-1.5B-Instruct2026.06 | 8 | 8 | 10 | 0 | 6.5 | |
| GRPOBackbone=Qwen2.5-1.5B-Instruct2026.06 | 0 | 0 | 0 | 0 | 0 | |
| HBGBackbone=Qwen2.5-1.5B-Instruct2026.06 | 0 | 0 | 0 | 0 | 0 |