Preference Alignment on Ultrafeedback 40% flipping ratio
78.87AccuracyFA-DPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FA-DPOBackbone=LLama-3.1-8B2025.11 | 78.87 | 68.5 | |
| FA-DPOBackbone=Mistral-7B2025.11 | 78.49 | 59 | |
| ROPOBackbone=LLama-3.1-8B2025.11 | 66.59 | 50.25 | |
| cDPOBackbone=LLama-3.1-8B2025.11 | 65.7 | 56.1 | |
| DPOBackbone=LLama-3.1-8B2025.11 | 64.96 | 56.1 | |
| ROPOBackbone=Mistral-7B2025.11 | 64.36 | 36.35 | |
| rDPOBackbone=LLama-3.1-8B2025.11 | 63.99 | 55.9 | |
| SIMPOBackbone=Mistral-7B2025.11 | 63.77 | 45 | |
| cDPOBackbone=Mistral-7B2025.11 | 63.24 | 34.6 | |
| rDPOBackbone=Mistral-7B2025.11 | 63.1 | 36.65 | |
| DPOBackbone=Mistral-7B2025.11 | 62.05 | 35.35 | |
| SIMPOBackbone=LLama-3.1-8B2025.11 | 57.07 | 38.95 |