Preference Optimization on Synthetic Color
85.66Win RateDPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DPOBackbone=Qwen2.5-3B-Instruct2025.06 | 85.66 | 0 | |
| F-betabeta (β)=10^0, Backbone=Qwen2.5-3B-Instruct2025.06 | 84.29 | 0.03 | |
| F-betabeta (β)=10^-2, Backbone=Qwen2.5-3B-Instruct2025.06 | 69.61 | 1.83 | |
| F-betabeta (β)=10^-4, Backbone=Qwen2.5-3B-Instruct2025.06 | 68.8 | 2.35 | |
| F-betabeta (β)=0, Backbone=Qwen2.5-3B-Instruct2025.06 | 61.57 | 8.83 |