Reward Modeling on Reward Bench safety subset prompt perturbations 2
-0.18EFLlama-3-OffsetBias-RM-8B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Llama-3-OffsetBias-RM-8BBackbone=Llama-3, Parameters=8B2025.11 | -0.18 | 0.065 | 0.179 | 0.063 | -0.003 | |
| FsfairX-LLaMA3-RM-v0.1Backbone=LLaMA3, Version=v0.12025.11 | -0.118 | 0.028 | 0.094 | 0.058 | -0.068 | |
| tulu-2-dpo-13bBackbone=tulu-2, Parameters=13b, Training=DPO2025.11 | -0.109 | 0.064 | 0.051 | -0.023 | 0.002 | |
| Skywork-Reward-Llama-3.1-8BBackbone=Llama-3.1, Parameters=8B2025.11 | -0.105 | -0.029 | -0.122 | -0.104 | 0.042 | |
| GRM-Llama3-8B-rewardmodel-ftBackbone=Llama3, Parameters=8B, Type=Reward Model FT2025.11 | -0.104 | 0.079 | -0.067 | -0.006 | 0.079 | |
| Qwen2.5-72B-InstructBackbone=Qwen2.5, Parameters=72B2025.11 | -0.101 | 0.341 | -0.007 | 0.188 | 0.222 | |
| URM-LLaMa-3-8BBackbone=LLaMa-3, Parameters=8B2025.11 | -0.07 | -0.018 | -0.026 | -0.192 | -0.01 | |
| Eurus-RM-7bBackbone=Eurus, Parameters=7b2025.11 | -0.068 | 0.087 | 0.061 | 0.084 | 0.016 | |
| SOLAR-10.7B-Instruct-v1.0Backbone=SOLAR, Parameters=10.7B, Version=v1.02025.11 | -0.052 | -0.084 | 0.009 | -0.031 | 0.01 | |
| GRM-llama3-8B-distillBackbone=Llama3, Parameters=8B, Type=Distill2025.11 | -0.051 | 0.001 | -0.023 | 0.041 | -0.015 | |
| URM-LLaMa-3.1-8BBackbone=LLaMa-3.1, Parameters=8B2025.11 | -0.029 | -0.05 | -0.042 | -0.121 | 0.069 | |
| Skywork-Reward-Llama-3.1-8B-v0.2Backbone=Llama-3.1, Parameters=8B, Version=v0.22025.11 | -0.023 | 0.012 | -0.038 | 0.031 | 0.134 | |
| Llama-3.1-70B-InstructBackbone=Llama-3.1, Parameters=70B2025.11 | -0.011 | 0.31 | 0.078 | 0.072 | 0.312 | |
| internlm2-20b-rewardBackbone=internlm2, Parameters=20b2025.11 | -0.009 | -0.225 | 0.101 | 0.008 | -0.029 | |
| Skywork-Reward-V2-Llama-3.1-8BBackbone=Llama-3.1, Parameters=8B, Version=V22025.11 | 0.014 | 0.073 | 0.063 | 0.062 | 0.044 | |
| Skywork-Reward-Gemma-2-27BBackbone=Gemma-2, Parameters=27B2025.11 | 0.029 | 0.069 | -0.033 | -0.071 | -0.05 | |
| GRM-llama3-8B-sftregBackbone=Llama3, Parameters=8B, Type=SFT-Reg2025.11 | 0.042 | 0.071 | 0.028 | 0.111 | -0.019 | |
| Starling-RM-34BParameters=34B2025.11 | 0.055 | 0.019 | 0.003 | 0.027 | 0.012 | |
| Skywork-Reward-V2-Qwen3-8BBackbone=Qwen3, Parameters=8B, Version=V22025.11 | 0.065 | 0.02 | 0.013 | -0.064 | 0.119 | |
| internlm2-1.8b-rewardBackbone=internlm2, Parameters=1.8B2025.11 | 0.092 | 0.001 | -0.128 | -0.173 | 0.009 | |
| Skywork-Reward-Gemma-2-27B-v0.2Backbone=Gemma-2, Parameters=27B, Version=v0.22025.11 | 0.126 | 0.057 | 0.064 | 0.076 | 0.073 | |
| Llama3-8B-IDRMBackbone=Llama3, Parameters=8B2025.11 | 0.16 | 0.276 | -0.055 | -0.169 | 0.061 | |
| Qwen3-8BBackbone=Qwen3, Parameters=8B2025.11 | 0.183 | -0.019 | -0.207 | 0.036 | 0.249 | |
| UltraRM-13bParameters=13b2025.11 | 0.213 | 0.1 | 0.046 | 0.292 | 0.074 | |
| internlm2-7b-rewardBackbone=internlm2, Parameters=7b2025.11 | 0.239 | -0.005 | 0.517 | 0.159 | 0.261 | |
| ArmoRM-Llama3-8B-v0.1Backbone=Llama3, Parameters=8B, Version=v0.12025.11 | 0.791 | -0.128 | -0.186 | -0.626 | 0.643 |