Preference Alignment on UFB
83.2Win RateCW-DPO
Evaluation Results
| Method | Links | |
|---|---|---|
| CW-DPOAlignment Algorithm=DPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 83.2 | |
| CW-rDPOAlignment Algorithm=rDPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 79.8 | |
| WS-DPOAlignment Algorithm=DPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 79.5 | |
| HumanAlignment Algorithm=IPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 77.1 | |
| HumanAlignment Algorithm=DPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 76.8 | |
| CW-IPOAlignment Algorithm=IPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 76.6 | |
| WS-DPOAlignment Algorithm=IPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 75.4 | |
| WS-DPOAlignment Algorithm=rDPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 74.3 | |
| UDPOBase Model=Llama-3.1-8B2026.07 | 74.3 | |
| α-UDPOBase Model=Llama-3.1-8B2026.07 | 73 | |
| IPOBase Model=Llama-3.1-8B2026.07 | 72.7 | |
| rDPOBase Model=Llama-3.1-8B2026.07 | 72.1 | |
| HumanAlignment Algorithm=rDPO, Model Pair=Qwen2.5-0.5B → Qwen2.5-14B2026.03 | 70.9 | |
| Dr.DPOBase Model=Llama-3.1-8B2026.07 | 65.8 | |
| DPOBase Model=Llama-3.1-8B2026.07 | 65.4 | |
| WS-DPOAlignment Algorithm=DPO, Model Pair=OPT-125M → OPT-13B2026.03 | 64.7 | |
| CW-DPOAlignment Algorithm=DPO, Model Pair=OPT-125M → OPT-13B2026.03 | 64.3 | |
| CW-IPOAlignment Algorithm=IPO, Model Pair=OPT-125M → OPT-13B2026.03 | 63.9 | |
| cDPOBase Model=Llama-3.1-8B2026.07 | 63.1 | |
| HumanAlignment Algorithm=IPO, Model Pair=OPT-125M → OPT-13B2026.03 | 62.1 | |
| α-UDPOBase Model=Qwen-3-8B2026.07 | 60.7 | |
| UDPOBase Model=Qwen-3-8B2026.07 | 60.6 | |
| WS-DPOAlignment Algorithm=IPO, Model Pair=OPT-125M → OPT-13B2026.03 | 60.2 | |
| HumanAlignment Algorithm=DPO, Model Pair=OPT-125M → OPT-13B2026.03 | 59.8 | |
| CW-rDPOAlignment Algorithm=rDPO, Model Pair=OPT-125M → OPT-13B2026.03 | 59.4 | |
| rDPOBase Model=Qwen-3-8B2026.07 | 58.3 | |
| IPOBase Model=Qwen-3-8B2026.07 | 57.8 | |
| WS-DPOAlignment Algorithm=rDPO, Model Pair=OPT-125M → OPT-13B2026.03 | 56.8 | |
| Dr.DPOBase Model=Qwen-3-8B2026.07 | 56.5 | |
| DPOBase Model=Qwen-3-8B2026.07 | 56.3 | |
| HumanAlignment Algorithm=rDPO, Model Pair=OPT-125M → OPT-13B2026.03 | 55.3 | |
| cDPOBase Model=Qwen-3-8B2026.07 | 55.1 |