Tool-based multi-turn dialogue on ToolWOZ (test)
0.922Avg Rewardgpt-4o-FC-JOSH-SFT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| gpt-4o-FC-JOSH-SFTBackbone=gpt-4o, Technique=Function Calling, Alignment Strategy=JOSH-SFT2024.09 | 0.922 | 84 | |
| gpt-4o-FCBackbone=gpt-4o, Technique=Function Calling2024.09 | 0.919 | 83.1 | |
| gpt-4o-ReACT-JOSH-SFTBackbone=gpt-4o, Technique=ReACT, Alignment Strategy=JOSH-SFT2024.09 | 0.914 | 81.3 | |
| gpt-4o-ReACTBackbone=gpt-4o, Technique=ReACT2024.09 | 0.9 | 79.1 | |
| gpt-4o-mini-FC-JOSH-SFTBackbone=gpt-4o-mini, Technique=Function Calling, Alignment Strategy=JOSH-SFT2024.09 | 0.89 | 78 | |
| gpt-4o-mini-FCBackbone=gpt-4o-mini, Technique=Function Calling2024.09 | 0.88 | 76 | |
| gpt-4o-mini-ReACT-JOSH-SFT-beam-16Backbone=gpt-4o-mini, Technique=ReACT, Alignment Strategy=JOSH-SFT, Decoding Strategy=beam-162024.09 | 0.865 | 74 | |
| gpt-4o-mini-ReACT-JOSH-SFT-beam-4Backbone=gpt-4o-mini, Technique=ReACT, Alignment Strategy=JOSH-SFT, Decoding Strategy=beam-42024.09 | 0.85 | 72 | |
| gpt-4o-mini-ReACT-JOSH-SFT-beam-8Backbone=gpt-4o-mini, Technique=ReACT, Alignment Strategy=JOSH-SFT, Decoding Strategy=beam-82024.09 | 0.85 | 72 | |
| meta-llama-3-8B-JOSH-KTOBackbone=meta-llama-3-8B, Technique=ReACT, Alignment Strategy=JOSH-KTO2024.09 | 0.79 | 59 | |
| gpt-3.5-FCBackbone=gpt-3.5, Technique=Function Calling2024.09 | 0.76 | 58 | |
| meta-llama-3-8B-JOSH-SFTBackbone=meta-llama-3-8B, Technique=ReACT, Alignment Strategy=JOSH-SFT2024.09 | 0.74 | 50 | |
| meta-llama-3-8B PPOBackbone=meta-llama-3-8B, Technique=ReACT, Alignment Strategy=PPO2024.09 | 0.69 | 45 | |
| gpt-4o-mini-ReACTBackbone=gpt-4o-mini, Technique=ReACT2024.09 | 0.67 | 48 | |
| gpt-3.5-ReACTBackbone=gpt-3.5, Technique=ReACT2024.09 | 0.66 | 44 | |
| meta-llama-3-8BBackbone=meta-llama-3-8B, Technique=ReACT2024.09 | 0.63 | 34 |