Alignment Reward Evaluation on HHA (test)
64Harmless ScoreSFT+DPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SFT+DPOBackbone=Llama3.2-3B2025.02 | 64 | 69.44 | 71.01 | 71.56 | 70.67 | 69 | |
| SFT+VARBackbone=Llama3.2-3B2025.02 | 64 | 67.93 | 71.32 | 70.83 | 70.02 | 68.52 | |
| SFT+VARBackbone=Llama3.1-8B2025.02 | 63.24 | 68.13 | 71.13 | 70.78 | 70.01 | 68.32 | |
| SFT+DPOBackbone=Llama3.1-8B2025.02 | 62.38 | 69.16 | 70 | 70.61 | 69.93 | 68.04 | |
| SFT+VARBackbone=Llama2-13B2025.02 | 61.29 | 63.27 | 66.07 | 65.75 | 65.03 | 64.09 | |
| SFT+VARBackbone=Llama3.2-1B2025.02 | 60.19 | 65.96 | 68.94 | 68.27 | 67.72 | 65.84 | |
| VARBackbone=Llama2-13B2025.02 | 58.45 | 58.94 | 62.94 | 61.89 | 61.26 | 60.56 | |
| VARBackbone=Llama3.2-3B2025.02 | 57.97 | 60.23 | 64.92 | 62.92 | 62.69 | 61.51 | |
| VARBackbone=Llama3.1-8B2025.02 | 57.18 | 61.13 | 65.57 | 64.33 | 63.68 | 62.06 | |
| SFT+DPOBackbone=Llama3.2-1B2025.02 | 56.43 | 64.65 | 64.95 | 65.9 | 65.16 | 62.98 | |
| DPOBackbone=Llama3.1-8B2025.02 | 56.17 | 59.89 | 61.02 | 60.86 | 60.59 | 59.48 | |
| SFT+DPOBackbone=Llama2-13B2025.02 | 55.19 | 59.9 | 60.61 | 61.26 | 60.59 | 59.24 | |
| DPOBackbone=Llama3.2-3B2025.02 | 53.71 | 59.38 | 60.04 | 60.55 | 59.99 | 58.42 | |
| VARBackbone=Llama3.2-1B2025.02 | 52.48 | 57.35 | 60.58 | 59.38 | 59.1 | 57.44 | |
| DPOBackbone=Llama2-13B2025.02 | 50.52 | 50.01 | 53.68 | 52.24 | 51.98 | 51.61 | |
| DPOBackbone=Llama3.2-1B2025.02 | 45.5 | 44.45 | 47.07 | 45.61 | 45.71 | 45.66 | |
| BaseBackbone=Llama3.1-8B2025.02 | 38.73 | 34.74 | 39.96 | 37.3 | 37.33 | 37.68 | |
| BaseBackbone=Llama3.2-1B2025.02 | 37.03 | 20.51 | 24.04 | 21.93 | 22.16 | 25.88 | |
| BaseBackbone=Llama3.2-3B2025.02 | 35.05 | 26.5 | 31.15 | 28.6 | 28.75 | 30.33 | |
| BaseBackbone=Llama2-13B2025.02 | 33.06 | 27.39 | 29.53 | 28.36 | 28.43 | 29.59 |