List your company
ResearchFormerly WizWand

Multi-Task evaluation on LongBench Chat

72.6Point-wise RateDPO w/ LongReward
59.70463.05266.469.748Oct 28, 2024
Updated 8mo ago

Evaluation results

MethodLinks
DPO w/ LongRewardBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1072.6
DPO w/ ContrastBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1070.6
SFTBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1069.8
DPO w/ LongRewardBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1069.2
officially post-trainedBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1068.6
DPO w/ ContrastBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1068.2
DPO w/ SRMBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1067.4
DPO w/ SRMBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1066.6
SFTBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1064.8
officially post-trainedBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.1060.2