Function Calling on BFCL Executable (test)
100Success Rate (Simple, Python)Prompting
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| PromptingBase Model=LLaMA-70B2025.03 | 100 | 91.4 | 96 | 84 | 82.5 | 89.6 | 90.8 | |
| PLAY2PROMPTBase Model=LLaMA-70B, Strategy=Prompting2025.03 | 100 | 91.4 | 98 | 88 | 95 | 94.2 | 94.5 | |
| PLAY2PROMPTBase Model=GPT-4o, Strategy=Function-calling2025.03 | 99 | 95.7 | 98 | 92 | 90 | 94.3 | 94.9 | |
| PLAY2PROMPTBase Model=GPT-3.5, Strategy=Function-calling2025.03 | 98 | 95.7 | 94 | 90 | 85 | 91.5 | 92.5 | |
| Function-callingBase Model=GPT-4o2025.03 | 98 | 98.6 | 94 | 94 | 77.5 | 91 | 92.4 | |
| PLAY2PROMPTBase Model=LLaMA-8B, Strategy=Prompting2025.03 | 97 | 87.1 | 96 | 92 | 92.5 | 93.1 | 92.9 | |
| Function-callingBase Model=GPT-3.52025.03 | 97 | 94.3 | 90 | 86 | 67.5 | 84.8 | 87 | |
| PromptingBase Model=LLaMA-8B2025.03 | 96 | 70 | 96 | 90 | 77.5 | 86.6 | 85.9 | |
| Function-callingBase Model=ToolACE-8B2025.03 | 96 | 92.9 | 92 | 86 | 70 | 85.6 | 87.4 | |
| Function-callingBase Model=Hammer-7B2025.03 | 96 | 72.9 | 88 | 86 | 70 | 82.1 | 82.6 | |
| PLAY2PROMPTBase Model=ToolACE-8B, Strategy=Function-calling2025.03 | 95 | 90 | 94 | 90 | 82.5 | 89.8 | 90.3 | |
| PLAY2PROMPTBase Model=Hammer-7B, Strategy=Function-calling2025.03 | 95 | 82.8 | 92 | 86 | 80 | 86.7 | 87.2 |