Multi-Task evaluation on LongBench Chat
72.6Point-wise RateDPO w/ LongReward
Evaluation Results
| Method | Links | |
|---|---|---|
| DPO w/ LongRewardBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 72.6 | |
| DPO w/ ContrastBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 70.6 | |
| SFTBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 69.8 | |
| DPO w/ LongRewardBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 69.2 | |
| officially post-trainedBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 68.6 | |
| DPO w/ ContrastBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 68.2 | |
| DPO w/ SRMBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 67.4 | |
| DPO w/ SRMBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 66.6 | |
| SFTBackbone=GLM-4-9B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 64.8 | |
| officially post-trainedBackbone=Llama-3.1-8B, Judge Model=GPT-4o, Prompting Protocol=few-shot2024.10 | 60.2 |