Reward Modeling on RewardBench Safety Subset Perturbations 2
-0.629LE ScoreLlama3-8B-IDRM
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Llama3-8B-IDRMBackbone=Llama3, Parameters=8B2025.11 | -0.629 | -0.7 | -0.032 | -0.119 | -0.73 | |
| Qwen2.5-72B-InstructBackbone=Qwen2.5, Parameters=72B2025.11 | -0.09 | 0.007 | 0.231 | -0.18 | 1.041 | |
| Starling-RM-34BParameters=34B2025.11 | -0.052 | -0.031 | -0.055 | -0.001 | -0.09 | |
| Qwen3-8BBackbone=Qwen3, Parameters=8B2025.11 | 0.002 | 0.148 | 0.191 | 1.201 | -0.004 | |
| Skywork-Reward-Gemma-2-27BBackbone=Gemma-2, Parameters=27B2025.11 | 0.029 | -0.015 | -0.003 | -0.027 | 0.016 | |
| SOLAR-10.7B-Instruct-v1.0Backbone=SOLAR, Parameters=10.7B, Version=v1.02025.11 | 0.107 | 0.046 | 0.002 | 0.007 | 0.004 | |
| internlm2-20b-rewardBackbone=internlm2, Parameters=20b2025.11 | 0.108 | 0.297 | 0.197 | 1.025 | -0.082 | |
| Skywork-Reward-Gemma-2-27B-v0.2Backbone=Gemma-2, Parameters=27B, Version=v0.22025.11 | 0.127 | 0.081 | 0.118 | 0.109 | 0.113 | |
| tulu-2-dpo-13bBackbone=tulu-2, Parameters=13b, Training=DPO2025.11 | 0.219 | 0.164 | 0.073 | 0.077 | 0.178 | |
| Llama-3.1-70B-InstructBackbone=Llama-3.1, Parameters=70B2025.11 | 0.344 | 0.179 | 0.169 | 1.47 | -0.037 | |
| ArmoRM-Llama3-8B-v0.1Backbone=Llama3, Parameters=8B, Version=v0.12025.11 | 0.388 | -0.297 | 0.821 | 1.016 | 1.066 | |
| Skywork-Reward-V2-Llama-3.1-8BBackbone=Llama-3.1, Parameters=8B, Version=V22025.11 | 0.502 | 0.567 | -0.003 | 0.092 | 0.613 | |
| Eurus-RM-7bBackbone=Eurus, Parameters=7b2025.11 | 0.535 | 0.558 | 0.13 | 0.02 | 0.525 | |
| internlm2-1.8b-rewardBackbone=internlm2, Parameters=1.8B2025.11 | 0.537 | 0.548 | 0.22 | 0.223 | 0.669 | |
| UltraRM-13bParameters=13b2025.11 | 0.537 | 0.582 | 0.25 | 0.154 | 0.58 | |
| Skywork-Reward-V2-Qwen3-8BBackbone=Qwen3, Parameters=8B, Version=V22025.11 | 0.543 | 0.558 | 0.12 | 0.127 | 0.628 | |
| Skywork-Reward-Llama-3.1-8BBackbone=Llama-3.1, Parameters=8B2025.11 | 0.571 | 0.605 | 0.062 | 0.057 | 0.668 | |
| GRM-llama3-8B-sftregBackbone=Llama3, Parameters=8B, Type=SFT-Reg2025.11 | 0.604 | 0.641 | 0.221 | 0.292 | 0.809 | |
| GRM-Llama3-8B-rewardmodel-ftBackbone=Llama3, Parameters=8B, Type=Reward Model FT2025.11 | 0.622 | 0.668 | 0.144 | 0.277 | 0.875 | |
| Skywork-Reward-Llama-3.1-8B-v0.2Backbone=Llama-3.1, Parameters=8B, Version=v0.22025.11 | 0.65 | 0.664 | 0.085 | 0.146 | 0.732 | |
| FsfairX-LLaMA3-RM-v0.1Backbone=LLaMA3, Version=v0.12025.11 | 0.694 | 0.728 | 0.338 | 0.412 | 0.952 | |
| GRM-llama3-8B-distillBackbone=Llama3, Parameters=8B, Type=Distill2025.11 | 0.701 | 0.663 | 0.242 | 0.427 | 0.873 | |
| URM-LLaMa-3.1-8BBackbone=LLaMa-3.1, Parameters=8B2025.11 | 0.726 | 0.771 | 0.059 | 0.107 | 0.791 | |
| internlm2-7b-rewardBackbone=internlm2, Parameters=7b2025.11 | 0.784 | 0.798 | 0.288 | 0.32 | 0.961 | |
| Llama-3-OffsetBias-RM-8BBackbone=Llama-3, Parameters=8B2025.11 | 0.928 | 0.93 | 0.201 | 0.359 | 1.039 | |
| URM-LLaMa-3-8BBackbone=LLaMa-3, Parameters=8B2025.11 | 1.112 | 1.05 | 0.563 | 0.725 | 1.264 |