Preference Alignment on Ultrafeedback (20% flipping ratio)
78.8AccuracyFA-DPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FA-DPOBackbone=LLama-3.1-8B2025.11 | 78.8 | 65.1 | |
| FA-DPOBackbone=Mistral-7B2025.11 | 78.05 | 61.35 | |
| ROPOBackbone=LLama-3.1-8B2025.11 | 75.82 | 62.75 | |
| cDPOBackbone=LLama-3.1-8B2025.11 | 75.22 | 64.5 | |
| rDPOBackbone=LLama-3.1-8B2025.11 | 73.96 | 59.75 | |
| ROPOBackbone=Mistral-7B2025.11 | 73.96 | 54.3 | |
| DPOBackbone=LLama-3.1-8B2025.11 | 73.89 | 62.9 | |
| cDPOBackbone=Mistral-7B2025.11 | 72.99 | 47.75 | |
| rDPOBackbone=Mistral-7B2025.11 | 71.95 | 44.75 | |
| DPOBackbone=Mistral-7B2025.11 | 71.35 | 45.75 | |
| SIMPOBackbone=Mistral-7B2025.11 | 67.04 | 54.75 | |
| SIMPOBackbone=LLama-3.1-8B2025.11 | 66.59 | 62.8 |