Function Calling on BFCL Individual Tools per Problem
95Execution AccuracyGold Documentation (Oracle)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Gold Documentation (Oracle)Model=GPT-5, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 95 | 96 | 96 | |
| Gold Documentation (Oracle)Model=GPT-5, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 95 | 96 | 96 | |
| Gold Documentation (Oracle)Model=GPT-5, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 95 | 96 | 96 | |
| Gold Documentation (Oracle)Model=GPT-5-mini, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 93 | 94 | 96 | |
| Gold Documentation (Oracle)Model=GPT-5-mini, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 93 | 94 | 96 | |
| Gold Documentation (Oracle)Model=GPT-5-mini, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 93 | 94 | 96 | |
| TOOLOBSERVERModel=GPT-5, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 86 | 82 | 91 | |
| TOOLOBSERVERModel=GPT-5-mini, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 85 | 89 | 96 | |
| TOOLOBSERVERModel=GPT-5-mini, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 84 | 81 | 91 | |
| Base (Opacified)Model=GPT-5-mini, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 84 | 88 | 96 | |
| TOOLOBSERVERModel=GPT-5, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 83 | 86 | 95 | |
| Base (Opacified)Model=GPT-5, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 82 | 85 | 96 | |
| TOOLOBSERVERModel=GPT-5, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 80 | 78 | 89 | |
| Play2PromptModel=GPT-5, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 79 | 81 | 95 | |
| Play2PromptModel=GPT-5-mini, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 77 | 80 | 95 | |
| EasyToolModel=GPT-5-mini, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 76 | 79 | 95 | |
| EasyToolModel=GPT-5, Documentation Setting=Anon. Fn. + Param Names, Prompting Strategy=ReAct2026.02 | 75 | 77 | 94 | |
| TOOLOBSERVERModel=GPT-5-mini, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 66 | 63 | 85 | |
| Play2PromptModel=GPT-5-mini, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 55 | 55 | 82 | |
| Play2PromptModel=GPT-5, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 53 | 54 | 82 | |
| Play2PromptModel=GPT-5, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 44 | 45 | 76 | |
| Play2PromptModel=GPT-5-mini, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 43 | 43 | 77 | |
| Base (Opacified)Model=GPT-5, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 | |
| EasyToolModel=GPT-5, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 | |
| Base (Opacified)Model=GPT-5-mini, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 | |
| EasyToolModel=GPT-5-mini, Documentation Setting=Anon. Fn. Names Only, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 | |
| Base (Opacified)Model=GPT-5, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 | |
| EasyToolModel=GPT-5, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 | |
| Base (Opacified)Model=GPT-5-mini, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 | |
| EasyToolModel=GPT-5-mini, Documentation Setting=Anon. Fn. + Real Desc, Prompting Strategy=ReAct2026.02 | 0 | 0 | 60 |