Reward Modeling on RM Bench Code
0.154EFSkywork-Reward-Gemma-2-27B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Skywork-Reward-Gemma-2-27BBackbone=Gemma-2, Size=27B2025.11 | 0.154 | 0.09 | 0.162 | 0.194 | 0.185 | 0.116 | 0.095 | 0.136 | 0.048 | 0.072 | |
| Skywork-Reward-Gemma-2-27Bparameters=27B2025.11 | 0.154 | 0.09 | 0.162 | 0.194 | 0.185 | 0.116 | 0.095 | 0.136 | 0.048 | 0.072 | |
| FsfairX-LLaMA3-RM-v0.1Backbone=LLaMA3, Version=v0.12025.11 | 0.132 | 0.033 | 0.088 | 0.022 | 0.047 | -0.013 | -0.028 | -0.016 | 0.051 | 0.125 | |
| FsfairX-LLaMA3-RM-v0.1parameters=8B2025.11 | 0.132 | 0.033 | 0.088 | 0.022 | 0.047 | -0.013 | -0.028 | -0.016 | 0.051 | 0.125 | |
| UltraRM-13bSize=13b2025.11 | 0.124 | 0.078 | 0.086 | 0.134 | 0.087 | 0.026 | 0.077 | 0.068 | 0.089 | 0.153 | |
| UltraRM-13bparameters=13B2025.11 | 0.124 | 0.078 | 0.086 | 0.134 | 0.087 | 0.026 | 0.077 | 0.068 | 0.089 | 0.153 | |
| URM-LLaMa-3.1-8BBackbone=LLaMa-3.1, Size=8B2025.11 | 0.113 | 0.111 | 0.058 | 0.121 | 0.028 | 0.064 | 0.039 | 0.022 | 0.111 | 0.156 | |
| URM-LLaMa-3.1-8Bparameters=8B2025.11 | 0.113 | 0.111 | 0.058 | 0.121 | 0.028 | 0.064 | 0.039 | 0.022 | 0.111 | 0.156 | |
| Skywork-Reward-V2-Llama-3.1-8BBackbone=Llama-3.1, Size=8B, Version=V22025.11 | 0.106 | 0.081 | 0.157 | 0.119 | 0.164 | -0.13 | -0.079 | 0.126 | 0.368 | 0.294 | |
| Skywork-Reward-V2-Llama-3.1-8Bparameters=8B2025.11 | 0.106 | 0.081 | 0.157 | 0.119 | 0.164 | -0.13 | -0.079 | 0.126 | 0.368 | 0.294 | |
| Llama-3.1-70B-InstructSize=70B, Type=Instruct2025.11 | 0.098 | -0.081 | 0.109 | 0.069 | 0.193 | 0.019 | -0.102 | 0.017 | 0.081 | -0.04 | |
| Llama-3.1-70B-Instructparameters=70B2025.11 | 0.098 | -0.081 | 0.109 | 0.069 | 0.193 | 0.019 | -0.102 | 0.017 | 0.081 | -0.04 | |
| GRM-llama3-8B-distillBackbone=llama3, Size=8B, Type=Distilled2025.11 | 0.095 | 0.011 | 0.19 | 0.096 | 0.089 | 0.05 | 0.051 | -0.047 | 0.023 | 0.142 | |
| GRM-llama3-8B-distillparameters=8B2025.11 | 0.095 | 0.011 | 0.19 | 0.096 | 0.089 | 0.05 | 0.051 | -0.047 | 0.023 | 0.142 | |
| Starling-RM-34BSize=34B, Type=Reward Model2025.11 | 0.084 | 0.028 | -0.066 | 0.039 | -0.032 | -0.083 | -0.174 | 0.013 | 0.023 | -0.099 | |
| Skywork-Reward-Gemma-2-27B-v0.2Backbone=Gemma-2, Size=27B, Version=v0.22025.11 | 0.084 | 0.079 | 0.05 | 0.055 | 0.178 | 0.109 | 0.088 | -0.037 | 0.022 | -0.018 | |
| Starling-RM-34Bparameters=34B2025.11 | 0.084 | 0.028 | -0.066 | 0.039 | -0.032 | -0.083 | -0.174 | 0.013 | 0.023 | -0.099 | |
| Skywork-Reward-Gemma-2-27B-v0.2parameters=27B2025.11 | 0.084 | 0.079 | 0.05 | 0.055 | 0.178 | 0.109 | 0.088 | -0.037 | 0.022 | -0.018 | |
| Skywork-Reward-Llama-3.1-8BBackbone=Llama-3.1, Size=8B2025.11 | 0.059 | -0.123 | 0.065 | 0.127 | 0.117 | 0.011 | 0.025 | 0.014 | 0.161 | 0.119 | |
| Skywork-Reward-Llama-3.1-8Bparameters=8B2025.11 | 0.059 | -0.123 | 0.065 | 0.127 | 0.117 | 0.011 | 0.025 | 0.014 | 0.161 | 0.119 | |
| GRM-llama3-8B-sftregBackbone=llama3, Size=8B, Type=SFT Regularized2025.11 | 0.049 | -0.025 | 0.115 | 0.069 | 0.069 | 0.013 | 0.009 | -0.056 | 0.047 | 0.141 | |
| GRM-llama3-8B-sftregparameters=8B2025.11 | 0.049 | -0.025 | 0.115 | 0.069 | 0.069 | 0.013 | 0.009 | -0.056 | 0.047 | 0.141 | |
| GRM-Llama3-8B-rewardmodel-ftBackbone=Llama3, Size=8B, Type=Reward Model Fine-Tuning2025.11 | 0.041 | -0.031 | -0.045 | -0.022 | -0.022 | -0.036 | -0.046 | -0.111 | 0 | 0.055 | |
| GRM-Llama3-8B-rewardmodel-ftparameters=8B2025.11 | 0.041 | -0.031 | -0.045 | -0.022 | -0.022 | -0.036 | -0.046 | -0.111 | 0 | 0.055 | |
| ArmoRM-Llama3-8B-v0.1Backbone=Llama3, Size=8B, Version=v0.12025.11 | 0.038 | 0.024 | 0.072 | 0.067 | 0.133 | 0.025 | -0.036 | -0.053 | -0.011 | 0.079 | |
| ArmoRM-Llama3-8B-v0.1parameters=8B2025.11 | 0.038 | 0.024 | 0.072 | 0.067 | 0.133 | 0.025 | -0.036 | -0.053 | -0.011 | 0.079 | |
| Skywork-Reward-Llama-3.1-8B-v0.2Backbone=Llama-3.1, Size=8B, Version=v0.22025.11 | 0.037 | -0.03 | 0.04 | 0.114 | -0.001 | 0.08 | 0.097 | -0.033 | 0.135 | 0.132 | |
| Skywork-Reward-Llama-3.1-8B-v0.2parameters=8B2025.11 | 0.037 | -0.03 | 0.04 | 0.114 | -0.001 | 0.08 | 0.097 | -0.033 | 0.135 | 0.132 | |
| internlm2-7b-rewardSize=7b, Type=Reward Model2025.11 | 0.035 | -0.004 | 0.056 | 0.115 | -0.047 | 0.012 | -0.033 | 0.009 | 0.03 | 0.009 | |
| internlm2-7b-rewardparameters=7B2025.11 | 0.035 | -0.004 | 0.056 | 0.115 | -0.047 | 0.012 | -0.033 | 0.009 | 0.03 | 0.009 | |
| URM-LLaMa-3-8BBackbone=LLaMa-3, Size=8B2025.11 | 0.03 | -0.038 | 0.02 | -0.05 | 0.031 | 0.035 | 0.058 | 0.02 | 0.04 | 0.147 | |
| URM-LLaMa-3-8Bparameters=8B2025.11 | 0.03 | -0.038 | 0.02 | -0.05 | 0.031 | 0.035 | 0.058 | 0.02 | 0.04 | 0.147 | |
| Qwen3-8BSize=8B2025.11 | 0.024 | 0.01 | 0.052 | -0.023 | 0.029 | -0.009 | 0.041 | 0.039 | 0.327 | -0.102 | |
| Qwen3-8Bparameters=8B2025.11 | 0.024 | 0.01 | 0.052 | -0.023 | 0.029 | -0.009 | 0.041 | 0.039 | 0.327 | -0.102 | |
| Llama-3-OffsetBias-RM-8BBackbone=Llama-3, Size=8B, Type=OffsetBias RM2025.11 | 0.022 | -0.028 | 0.071 | -0.004 | 0.072 | -0.046 | -0.086 | -0.024 | 0.059 | 0.093 | |
| Llama-3-OffsetBias-RM-8Bparameters=8B2025.11 | 0.022 | -0.028 | 0.071 | -0.004 | 0.072 | -0.046 | -0.086 | -0.024 | 0.059 | 0.093 | |
| Eurus-RM-7bSize=7b, Type=Reward Model2025.11 | 0.007 | -0.017 | -0.094 | 0.006 | -0.084 | -0.042 | -0.082 | -0.013 | 0.1 | 0.064 | |
| Eurus-RM-7bparameters=7B2025.11 | 0.007 | -0.017 | -0.094 | 0.006 | -0.084 | -0.042 | -0.082 | -0.013 | 0.1 | 0.064 | |
| Qwen2.5-72B-InstructSize=72B, Type=Instruct2025.11 | -0.015 | 0.023 | -0.08 | 0.109 | -0.125 | 0.078 | -0.098 | 0.044 | 0.23 | 0.066 | |
| Qwen2.5-72B-Instructparameters=72B2025.11 | -0.015 | 0.023 | -0.08 | 0.109 | -0.125 | 0.078 | -0.098 | 0.044 | 0.23 | 0.066 | |
| SOLAR-10.7B-Instruct-v1.0Size=10.7B, Version=v1.0, Type=Instruct2025.11 | -0.026 | -0.035 | -0.001 | -0.073 | -0.04 | 0.019 | 0.027 | -0.08 | -0.028 | 0.01 | |
| SOLAR-10.7B-Instruct-v1.0parameters=10.7B2025.11 | -0.026 | -0.035 | -0.001 | -0.073 | -0.04 | 0.019 | 0.027 | -0.08 | -0.028 | 0.01 | |
| internlm2-20b-rewardSize=20b, Type=Reward Model2025.11 | -0.03 | -0.035 | 0.079 | 0.032 | 0.083 | -0.015 | -0.037 | 0.122 | 0.112 | 0.013 | |
| internlm2-20b-rewardparameters=20B2025.11 | -0.03 | -0.035 | 0.079 | 0.032 | 0.083 | -0.015 | -0.037 | 0.122 | 0.112 | 0.013 | |
| Llama3-8B-IDRMBackbone=Llama3, Size=8B, Type=IDRM2025.11 | -0.051 | -0.02 | 0.105 | 0.082 | 0.017 | -0.082 | -0.071 | -0.04 | -0.059 | -0.072 | |
| Llama3-8B-IDRMparameters=8B2025.11 | -0.051 | -0.02 | 0.105 | 0.082 | 0.017 | -0.082 | -0.071 | -0.04 | -0.059 | -0.072 | |
| tulu-2-dpo-13bSize=13b, Type=DPO2025.11 | -0.073 | 0.017 | -0.089 | -0.07 | -0.017 | -0.091 | -0.016 | -0.031 | -0.066 | 0.063 | |
| tulu-2-dpo-13bparameters=13B2025.11 | -0.073 | 0.017 | -0.089 | -0.07 | -0.017 | -0.091 | -0.016 | -0.031 | -0.066 | 0.063 | |
| Skywork-Reward-V2-Qwen3-8BBackbone=Qwen3, Size=8B, Version=V22025.11 | -0.138 | -0.029 | 0.064 | 0.056 | 0.118 | -0.019 | -0.037 | 0.052 | 0.218 | 0.209 | |
| Skywork-Reward-V2-Qwen3-8Bparameters=8B2025.11 | -0.138 | -0.029 | 0.064 | 0.056 | 0.118 | -0.019 | -0.037 | 0.052 | 0.218 | 0.209 | |
| internlm2-1.8b-rewardSize=1.8b, Type=Reward Model2025.11 | -0.14 | -0.125 | -0.063 | -0.05 | 0.001 | 0.041 | 0.039 | -0.049 | 0.033 | 0.05 | |
| internlm2-1.8b-rewardparameters=1.8B2025.11 | -0.14 | -0.125 | -0.063 | -0.05 | 0.001 | 0.041 | 0.039 | -0.049 | 0.033 | 0.05 |