Dialogue Preference Learning on HH (test)
90.8Win Rate (0% Flip)UDPO
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| UDPOModel=Llama-3.2-3B2026.07 | 90.8 | 90.4 | 85.6 | 85.9 | |
| α-UDPOModel=Llama-3.2-3B2026.07 | 90.4 | 89.8 | 85.7 | 86.1 | |
| Dr.DPOModel=Llama-3.2-3B2026.07 | 88.2 | 88.2 | 81.6 | 85 | |
| rDPOModel=Llama-3.2-3B2026.07 | 88 | 89.3 | 84.4 | 84.7 | |
| IPOModel=Llama-3.2-3B2026.07 | 85.6 | 89.3 | 80 | 81.8 | |
| DPOModel=Llama-3.2-3B2026.07 | 85.4 | 80.4 | 69.8 | 71.4 | |
| cDPOModel=Llama-3.2-3B2026.07 | 85.3 | 80.2 | 70.9 | 70.7 | |
| UDPOModel=Qwen-3-1.7B2026.07 | 83.4 | 81.2 | 73 | 79.4 | |
| α-UDPOModel=Qwen-3-1.7B2026.07 | 81.3 | 80.3 | 73.2 | 79.6 | |
| Dr.DPOModel=Qwen-3-1.7B2026.07 | 80.2 | 77.7 | 68.2 | 76.6 | |
| rDPOModel=Qwen-3-1.7B2026.07 | 78.8 | 72.9 | 68.1 | 74.5 | |
| DPOModel=Qwen-3-1.7B2026.07 | 75.4 | 68.2 | 58.2 | 65.9 | |
| cDPOModel=Qwen-3-1.7B2026.07 | 72.8 | 70.1 | 59.3 | 63.6 | |
| IPOModel=Qwen-3-1.7B2026.07 | 71.7 | 68.7 | 63.3 | 72 |