Multi-turn tool calling on BFCL MT v4
48Success RatePass@N (oracle)
Evaluation results
| Method | Links | |
|---|---|---|
| Pass@N (oracle)Training=false, Backbone=Qwen3.5-9B2026.06 | 48 | |
| Greedy DecodingTraining=false, Backbone=Qwen3.5-9B2026.06 | 42.5 | |
| WildReward-8BTraining=true, Backbone=Qwen3.5-9B2026.06 | 42.5 | |
| Progress AdvantageTraining=false, Backbone=Qwen3.5-9B2026.06 | 42.5 | |
| ThinkPRM-14BTraining=true, Backbone=Qwen3.5-9B2026.06 | 40 | |
| Mean-of-NTraining=false, Backbone=Qwen3.5-9B2026.06 | 38.5 | |
| ThinkPRM-7BTraining=true, Backbone=Qwen3.5-9B2026.06 | 37 | |
| DeepConf TailTraining=false, Backbone=Qwen3.5-9B2026.06 | 36 | |
| Self-CertaintyTraining=false, Backbone=Qwen3.5-9B2026.06 | 34.5 | |
| DeepConf B10Training=false, Backbone=Qwen3.5-9B2026.06 | 34.5 | |
| Pass@N (oracle)Training=false, Backbone=Gemma4-4B2026.06 | 22 | |
| WildReward-8BTraining=true, Backbone=Gemma4-4B2026.06 | 20 | |
| Greedy DecodingTraining=false, Backbone=Gemma4-4B2026.06 | 19 | |
| ThinkPRM-14BTraining=true, Backbone=Gemma4-4B2026.06 | 19 | |
| Progress AdvantageTraining=false, Backbone=Gemma4-4B2026.06 | 19 | |
| ThinkPRM-7BTraining=true, Backbone=Gemma4-4B2026.06 | 18.5 | |
| Mean-of-NTraining=false, Backbone=Gemma4-4B2026.06 | 17.5 | |
| DeepConf TailTraining=false, Backbone=Gemma4-4B2026.06 | 15.5 | |
| DeepConf B10Training=false, Backbone=Gemma4-4B2026.06 | 15.5 | |
| Self-CertaintyTraining=false, Backbone=Gemma4-4B2026.06 | 15 |