LLM Alignment on UltraFeedback (test)
21AlpacaEval 2 Win Rate (WR)PD (ours)
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| PD (ours)Model=Llama3.1-8B, Category=SELT2025.08 | 21 | 26.11 | 1.24 | 18.6 | |
| RAFModel=Llama3.1-8B, Category=SELT2025.08 | 19.64 | 23.34 | 1.18 | 16.2 | |
| DMPOModel=Llama3.1-8B, Category=FULL2025.08 | 19.42 | 24.73 | 1.25 | 42.1 | |
| PD (rati.)Model=Llama3.1-8B, Category=SELT2025.08 | 18.85 | 25.38 | 1.23 | 10.1 | |
| AVG.Model=Llama3.1-8B, Category=FULL2025.08 | 16.63 | 22.21 | 1.15 | 33.6 | |
| ALLModel=Llama3.1-8B, Category=FULL2025.08 | 15.18 | 21.14 | 1.08 | 33.6 | |
| RANDModel=Llama3.1-8B, Category=SELT2025.08 | 14.72 | 19.56 | 1.02 | 10.1 | |
| OVA.Model=Llama3.1-8B, Category=FULL2025.08 | 14.35 | 19.96 | 1 | 33.6 | |
| PDModel=Qwen2.5-7B, Strategy Group=SELT2025.08 | 13.81 | 22.17 | — | — | |
| DMPOModel=Qwen2.5-7B, Strategy Group=FULL2025.08 | 12.77 | 18.42 | — | — | |
| RAFModel=Qwen2.5-7B, Strategy Group=SELT2025.08 | 12.01 | 18.06 | — | — | |
| PD (rati.)Model=Qwen2.5-7B, Strategy Group=SELT2025.08 | 10.53 | 17.03 | — | — | |
| AVG.Model=Qwen2.5-7B, Strategy Group=FULL2025.08 | 9.73 | 15.74 | — | — | |
| RANDModel=Qwen2.5-7B, Strategy Group=SELT2025.08 | 9.35 | 16.03 | — | — | |
| ALLModel=Qwen2.5-7B, Strategy Group=FULL2025.08 | 9.33 | 15.6 | — | — | |
| OVA.Model=Qwen2.5-7B, Strategy Group=FULL2025.08 | 8.11 | 16.47 | — | — | |
| SFTModel=Llama3.1-8B, Category=INIT2025.08 | 7.08 | 14 | 0.64 | 0 | |
| SFTModel=Qwen2.5-7B, Strategy Group=INIT2025.08 | 4.36 | 9.12 | — | — | |
| ALLModel=Qwen2.5-7B, Strategy Group=FULL2025.08 | — | — | 1.15 | — | |
| AVG.Model=Qwen2.5-7B, Strategy Group=FULL2025.08 | — | — | 1.2 | — | |
| DMPOModel=Qwen2.5-7B, Strategy Group=FULL2025.08 | — | — | 1.33 | — | |
| OVA.Model=Qwen2.5-7B, Strategy Group=FULL2025.08 | — | — | 1 | — | |
| PDModel=Qwen2.5-7B, Strategy Group=SELT2025.08 | — | — | 1.3 | — | |
| PD (rati.)Model=Qwen2.5-7B, Strategy Group=SELT2025.08 | — | — | 1.22 | — | |
| RAFModel=Qwen2.5-7B, Strategy Group=SELT2025.08 | — | — | 1.28 | — | |
| RANDModel=Qwen2.5-7B, Strategy Group=SELT2025.08 | — | — | 1.15 | — | |
| SFTModel=Qwen2.5-7B, Strategy Group=INIT2025.08 | — | — | 0.77 | — |