List your company
ResearchFormerly WizWand

Tool-based multi-turn dialogue on ToolWOZ (test)

0.922Avg Rewardgpt-4o-FC-JOSH-SFT
0.618320.697160.7760.85484Sep 6, 2024
Updated 8mo ago

Evaluation results

MethodLinks
gpt-4o-FC-JOSH-SFTBackbone=gpt-4o, Technique=Function Calling, Alignment Strategy=JOSH-SFT2024.090.92284
gpt-4o-FCBackbone=gpt-4o, Technique=Function Calling2024.090.91983.1
gpt-4o-ReACT-JOSH-SFTBackbone=gpt-4o, Technique=ReACT, Alignment Strategy=JOSH-SFT2024.090.91481.3
gpt-4o-ReACTBackbone=gpt-4o, Technique=ReACT2024.090.979.1
gpt-4o-mini-FC-JOSH-SFTBackbone=gpt-4o-mini, Technique=Function Calling, Alignment Strategy=JOSH-SFT2024.090.8978
gpt-4o-mini-FCBackbone=gpt-4o-mini, Technique=Function Calling2024.090.8876
gpt-4o-mini-ReACT-JOSH-SFT-beam-16Backbone=gpt-4o-mini, Technique=ReACT, Alignment Strategy=JOSH-SFT, Decoding Strategy=beam-162024.090.86574
gpt-4o-mini-ReACT-JOSH-SFT-beam-4Backbone=gpt-4o-mini, Technique=ReACT, Alignment Strategy=JOSH-SFT, Decoding Strategy=beam-42024.090.8572
gpt-4o-mini-ReACT-JOSH-SFT-beam-8Backbone=gpt-4o-mini, Technique=ReACT, Alignment Strategy=JOSH-SFT, Decoding Strategy=beam-82024.090.8572
meta-llama-3-8B-JOSH-KTOBackbone=meta-llama-3-8B, Technique=ReACT, Alignment Strategy=JOSH-KTO2024.090.7959
gpt-3.5-FCBackbone=gpt-3.5, Technique=Function Calling2024.090.7658
meta-llama-3-8B-JOSH-SFTBackbone=meta-llama-3-8B, Technique=ReACT, Alignment Strategy=JOSH-SFT2024.090.7450
meta-llama-3-8B PPOBackbone=meta-llama-3-8B, Technique=ReACT, Alignment Strategy=PPO2024.090.6945
gpt-4o-mini-ReACTBackbone=gpt-4o-mini, Technique=ReACT2024.090.6748
gpt-3.5-ReACTBackbone=gpt-3.5, Technique=ReACT2024.090.6644
meta-llama-3-8BBackbone=meta-llama-3-8B, Technique=ReACT2024.090.6334