RLHF Alignment Evaluation on HHA
76.1Harmlessness Win Rate (Base, A)DPO
Evaluation Results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DPOBackbone=Llama3.2-3B2025.02 | 76.1 | 20.54 | 3.37 | 76.77 | 14.81 | 8.42 | 49.83 | 44.1 | 6.06 | 62.97 | 25.59 | 11.45 | 66.42 | 63.19 | |
| VARBackbone=Llama3.2-3B2025.02 | 63.3 | 26.26 | 10.44 | 89.9 | 3.7 | 6.4 | 61.96 | 32.66 | 5.39 | 79.13 | 12.12 | 8.75 | 73.57 | 76.99 | |
| VARBackbone=Llama3.2-1B2025.02 | 55.56 | 34.34 | 10.1 | 72.73 | 16.5 | 10.77 | 48.48 | 42.42 | 9.09 | 69.36 | 19.86 | 10.77 | 61.53 | 63.53 | |
| VARBackbone=Llama3.1-8B2025.02 | 51.52 | 33.67 | 14.81 | 88.55 | 8.08 | 3.37 | 58.59 | 34.34 | 7.07 | 85.86 | 10.44 | 3.7 | 71.13 | 77.67 | |
| DPOBackbone=Llama3.1-8B2025.02 | 42.42 | 42.42 | 15.15 | 68.69 | 19.19 | 12.12 | 33.33 | 54.21 | 12.46 | 53.2 | 31.65 | 15.15 | 49.41 | 51.74 | |
| DPOBackbone=Llama3.2-1B2025.02 | 20.2 | 59.6 | 20.2 | 49.49 | 32.32 | 18.18 | 12.46 | 71.38 | 16.16 | 34.68 | 47.47 | 17.84 | 29.21 | 32.21 |