Reward Scoring on HHA benchmark
66.97Harmlessness Score (Base)SFT+DPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SFT+DPOModel=Qwen2.5-14B2025.02 | 66.97 | 67.83 | 68.16 | 68.94 | 68.31 | 67.97 | |
| SFT+VARModel=Qwen2.5-14B2025.02 | 66.37 | 67.74 | 71.7 | 71.08 | 70.17 | 69.22 | |
| SFT+VARModel=Qwen2.5-3B2025.02 | 65.15 | 67.86 | 71.24 | 70.78 | 69.96 | 68.75 | |
| SFT+VARModel=Qwen2.5-7B2025.02 | 64.96 | 66.46 | 69.81 | 69.39 | 68.55 | 67.65 | |
| VARModel=Qwen2.5-7B2025.02 | 64.86 | 66.41 | 70.87 | 69.78 | 69.02 | 67.98 | |
| VARModel=Qwen2.5-14B2025.02 | 64.24 | 65.23 | 69.79 | 68.5 | 67.84 | 66.94 | |
| VARModel=Qwen2.5-3B2025.02 | 63.3 | 66.29 | 70.07 | 69.32 | 68.56 | 67.24 | |
| SFT+VARModel=Qwen2.5-1.5B2025.02 | 62.76 | 66.07 | 69.13 | 68.78 | 67.99 | 66.69 | |
| DPOModel=Qwen2.5-7B2025.02 | 61.51 | 66.09 | 66.5 | 67.28 | 66.62 | 65.35 | |
| VARModel=Qwen2.5-1.5B2025.02 | 61.33 | 64.72 | 68.87 | 68.01 | 67.2 | 65.73 | |
| SFT+DPOModel=Qwen2.5-7B2025.02 | 60.78 | 62.02 | 61.62 | 62.67 | 62.1 | 61.77 | |
| DPOModel=Qwen2.5-3B2025.02 | 60.58 | 63.37 | 63.84 | 64.9 | 64.03 | 63.17 | |
| SFT+VARModel=Qwen2.5-0.5B2025.02 | 58.22 | 61.72 | 63.58 | 63.56 | 62.95 | 61.77 | |
| SFT+DPOModel=Qwen2.5-0.5B2025.02 | 56.42 | 58.02 | 60.38 | 59.91 | 59.44 | 58.68 | |
| DPOModel=Qwen2.5-14B2025.02 | 55.83 | 55.63 | 59.04 | 57.61 | 57.43 | 57.03 | |
| VARModel=Qwen2.5-0.5B2025.02 | 55.22 | 58.09 | 62.32 | 60.38 | 60.26 | 59 | |
| DPOModel=Qwen2.5-0.5B2025.02 | 55.21 | 55.5 | 56.24 | 56.75 | 56.17 | 55.93 | |
| SFT+DPOModel=Qwen2.5-3B2025.02 | 54.61 | 58.45 | 58.69 | 60.19 | 59.11 | 57.98 | |
| SFT+DPOModel=Qwen2.5-1.5B2025.02 | 54.51 | 61.13 | 61.54 | 62.26 | 61.64 | 59.86 | |
| DPOModel=Qwen2.5-1.5B2025.02 | 53.4 | 57 | 57.38 | 57.85 | 57.41 | 56.41 | |
| SFT+VARModel=Qwen2.5-32B-Int42025.02 | 53.18 | 49.07 | 55.66 | 51.9 | 52.21 | 52.45 | |
| VARModel=Qwen2.5-32B-Int42025.02 | 50.03 | 45.36 | 51.77 | 47.77 | 48.3 | 48.73 | |
| BaseModel=Qwen2.5-3B2025.02 | 47.07 | 34.27 | 41.86 | 36.61 | 37.58 | 39.96 | |
| BaseModel=Qwen2.5-7B2025.02 | 42.49 | 42.02 | 47.96 | 44.21 | 44.73 | 44.17 | |
| BaseModel=Qwen2.5-14B2025.02 | 39.29 | 32.73 | 39.03 | 35.16 | 35.64 | 36.55 | |
| BaseModel=Qwen2.5-32B-Int42025.02 | 38.8 | 34.36 | 39.13 | 36.78 | 36.77 | 37.27 | |
| SFT+DPOModel=Qwen2.5-32B-Int42025.02 | 37.95 | 27.7 | 28.63 | 28.16 | 28.16 | 30.61 | |
| DPOModel=Qwen2.5-32B-Int42025.02 | 37.09 | 31.38 | 34.58 | 33.04 | 33 | 34.02 | |
| BaseModel=Qwen2.5-1.5B2025.02 | 35.01 | 26.18 | 32.11 | 28.13 | 28.81 | 30.36 | |
| BaseModel=Qwen2.5-0.5B2025.02 | 33.03 | 25.44 | 30.86 | 26.94 | 27.75 | 29.06 |