Alignment Reward Evaluation on OffsetBias (test)
67.94RewardSFT+DPO
Evaluation results
| Method | Links | |
|---|---|---|
| SFT+DPOModel=Qwen2.5-14B2025.02 | 67.94 | |
| SFT+VARModel=Qwen2.5-14B2025.02 | 66.44 | |
| VARModel=Qwen2.5-3B2025.02 | 66.06 | |
| SFT+VARModel=Qwen2.5-1.5B2025.02 | 65.88 | |
| DPOModel=Qwen2.5-7B2025.02 | 65.81 | |
| SFT+VARBackbone=Llama3.1-8B2025.02 | 65.75 | |
| SFT+VARModel=Qwen2.5-3B2025.02 | 65.63 | |
| VARModel=Qwen2.5-14B2025.02 | 65.63 | |
| VARModel=Qwen2.5-7B2025.02 | 65.38 | |
| SFT+VARModel=Qwen2.5-7B2025.02 | 65.38 | |
| VARModel=Qwen2.5-1.5B2025.02 | 64.75 | |
| VARBackbone=Llama3.1-8B2025.02 | 63.91 | |
| SFT+DPOBackbone=Llama3.2-3B2025.02 | 63.81 | |
| SFT+VARBackbone=Llama3.2-3B2025.02 | 63.72 | |
| DPOModel=Qwen2.5-14B2025.02 | 62.94 | |
| SFT+VARBackbone=Llama2-13B2025.02 | 62.59 | |
| SFT+VARBackbone=Llama3.2-1B2025.02 | 61.97 | |
| VARBackbone=Llama2-13B2025.02 | 61.44 | |
| VARBackbone=Llama3.2-3B2025.02 | 60.88 | |
| SFT+DPOBackbone=Llama3.1-8B2025.02 | 60.88 | |
| SFT+VARModel=Qwen2.5-0.5B2025.02 | 60.63 | |
| VARModel=Qwen2.5-0.5B2025.02 | 59.5 | |
| SFT+DPOBackbone=Llama3.2-1B2025.02 | 59.09 | |
| SFT+DPOBackbone=Llama2-13B2025.02 | 59.09 | |
| SFT+VARModel=Qwen2.5-32B-Int42025.02 | 59.09 | |
| DPOModel=Qwen2.5-3B2025.02 | 58.97 | |
| VARModel=Qwen2.5-32B-Int42025.02 | 57.69 | |
| SFT+DPOModel=Qwen2.5-1.5B2025.02 | 57.63 | |
| SFT+DPOModel=Qwen2.5-7B2025.02 | 56.91 | |
| VARBackbone=Llama3.2-1B2025.02 | 56.81 | |
| DPOModel=Qwen2.5-1.5B2025.02 | 56.63 | |
| SFT+DPOModel=Qwen2.5-0.5B2025.02 | 55.88 | |
| BaseModel=Qwen2.5-7B2025.02 | 54.09 | |
| DPOBackbone=Llama3.2-3B2025.02 | 53.94 | |
| DPOModel=Qwen2.5-0.5B2025.02 | 53.44 | |
| SFT+DPOModel=Qwen2.5-3B2025.02 | 52.63 | |
| DPOBackbone=Llama2-13B2025.02 | 51.75 | |
| DPOBackbone=Llama3.1-8B2025.02 | 51.38 | |
| BaseModel=Qwen2.5-14B2025.02 | 45.16 | |
| BaseModel=Qwen2.5-3B2025.02 | 45.06 | |
| BaseModel=Qwen2.5-0.5B2025.02 | 40.38 | |
| BaseModel=Qwen2.5-32B-Int42025.02 | 38.97 | |
| DPOBackbone=Llama3.2-1B2025.02 | 37.31 | |
| DPOModel=Qwen2.5-32B-Int42025.02 | 32.38 | |
| BaseBackbone=Llama3.1-8B2025.02 | 30.42 | |
| SFT+DPOModel=Qwen2.5-32B-Int42025.02 | 30.23 | |
| BaseBackbone=Llama2-13B2025.02 | 27.05 | |
| BaseBackbone=Llama3.2-3B2025.02 | 26.61 | |
| BaseModel=Qwen2.5-1.5B2025.02 | 26.52 | |
| BaseBackbone=Llama3.2-1B2025.02 | 21 |