Tool-use performance on Tau-bench Retail (test)
66Pass Rategpt-4o-FC-JOSH-SFT
Evaluation Results
| Method | Links | |
|---|---|---|
| gpt-4o-FC-JOSH-SFTBackbone=gpt-4o, Prompting=Function Calling (FC), Training=JOSH-SFT2024.09 | 66 | |
| gpt-4o-FCBackbone=gpt-4o, Prompting=Function Calling (FC)2024.09 | 65.21 | |
| gpt-4o-ACT-JOSH-SFTBackbone=gpt-4o, Prompting=ACT, Training=JOSH-SFT2024.09 | 64.26 | |
| gpt-4o-ACTBackbone=gpt-4o, Prompting=ACT2024.09 | 63.13 | |
| gpt-4o-ReACT-JOSH-SFTBackbone=gpt-4o, Prompting=ReACT, Training=JOSH-SFT2024.09 | 58.43 | |
| gpt-4o-mini-FC-JOSH-SFTBackbone=gpt-4o-mini, Prompting=Function Calling (FC), Training=JOSH-SFT2024.09 | 58.26 | |
| gpt-4o-ReACTBackbone=gpt-4o, Prompting=ReACT2024.09 | 54.43 | |
| gpt-4o-mini-FCBackbone=gpt-4o-mini, Prompting=Function Calling (FC)2024.09 | 50.78 | |
| gpt-4o-mini-ACT-JOSH-SFTBackbone=gpt-4o-mini, Prompting=ACT, Training=JOSH-SFT2024.09 | 47.65 | |
| gpt-4o-mini-ACTBackbone=gpt-4o-mini, Prompting=ACT2024.09 | 44.6 | |
| gpt-4o-mini-ReACT-JOSH-SFTBackbone=gpt-4o-mini, Prompting=ReACT, Training=JOSH-SFT2024.09 | 36.34 | |
| gpt-4o-mini-ReACTBackbone=gpt-4o-mini, Prompting=ReACT2024.09 | 16.87 |