Preference Optimization on Alpaca-GPT4 (Expertise)
76.97Win RateDPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DPOBackbone=Qwen2.5-3B-Instruct2025.06 | 76.97 | 13.21 | |
| F-betabeta (β)=10^0, Backbone=Qwen2.5-3B-Instruct2025.06 | 76.36 | 12.73 | |
| F-betabeta (β)=10^-2, Backbone=Qwen2.5-3B-Instruct2025.06 | 76.34 | 13.92 | |
| F-betabeta (β)=0, Backbone=Qwen2.5-3B-Instruct2025.06 | 76.13 | 14.28 | |
| F-betabeta (β)=10^-4, Backbone=Qwen2.5-3B-Instruct2025.06 | 76.1 | 14.18 |