Reward Modeling on RM-Bench Hard
0.7091Average AccuracyHARVE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| HARVEBackbone=Skywork-V2-Llama-3.2 (3B), alpha=1.252026.06 | 0.7091 | — | 0.6434 | 0.5058 | 0.6276 | 0.9894 | 0.8259 | |
| BaselineBackbone=Skywork-V2-Llama-3.2 (3B)2026.06 | 0.6943 | — | 0.5504 | 0.519 | 0.6251 | 0.9754 | 0.7919 | |
| Finetune (3:1)Backbone=Skywork-V2-Llama-3.2 (3B), ratio=3:12026.06 | 0.694 | — | 0.553 | 0.5219 | 0.6226 | 0.9754 | 0.7919 | |
| Finetune (5:1)Backbone=Skywork-V2-Llama-3.2 (3B), ratio=5:12026.06 | 0.6933 | — | 0.5504 | 0.519 | 0.6226 | 0.9754 | 0.7919 | |
| HARVEBackbone=RM-Mistral (7B), alpha=2.502026.06 | 0.6506 | — | 0.5375 | 0.4927 | 0.5293 | 0.9425 | 0.8535 | |
| HARVEBackbone=InternLM2-reward (20B), alpha=1.602026.06 | 0.6378 | — | 0.354 | 0.3553 | 0.6698 | 0.9589 | 0.5924 | |
| BaselineBackbone=InternLM2-reward (20B)2026.06 | 0.6265 | — | 0.292 | 0.3582 | 0.6786 | 0.9566 | 0.518 | |
| Finetune (3:1)Backbone=InternLM2-reward (20B), ratio=3:12026.06 | 0.6257 | — | 0.292 | 0.3596 | 0.6761 | 0.9566 | 0.518 | |
| Finetune (5:1)Backbone=InternLM2-reward (20B), ratio=5:12026.06 | 0.6247 | — | 0.2868 | 0.3567 | 0.6736 | 0.9577 | 0.5244 | |
| HARVEBackbone=Skywork-V2-Qwen3 (0.6B), alpha=1.252026.06 | 0.6204 | — | 0.4987 | 0.3713 | 0.5766 | 0.9601 | 0.6157 | |
| HARVEBackbone=Skywork-Reward-Llama-3.1 (8B), alpha=1.102026.06 | 0.6122 | — | 0.509 | 0.4518 | 0.4304 | 0.9859 | 0.8662 | |
| BaselineBackbone=InternLM2-reward (1.8B)2026.06 | 0.5956 | — | 0.2429 | 0.2515 | 0.6969 | 0.9261 | 0.4459 | |
| HARVEBackbone=InternLM2-reward (1.8B), alpha=1.102026.06 | 0.5948 | — | 0.199 | 0.3743 | 0.6824 | 0.9014 | 0.3907 | |
| Finetune (3:1)Backbone=InternLM2-reward (1.8B), ratio=3:12026.06 | 0.5933 | — | 0.2403 | 0.2456 | 0.6944 | 0.9261 | 0.4459 | |
| Finetune (5:1)Backbone=InternLM2-reward (1.8B), ratio=5:12026.06 | 0.5896 | — | 0.2351 | 0.2383 | 0.6875 | 0.9284 | 0.448 | |
| Finetune (3:1)Backbone=Skywork-V2-Qwen3 (0.6B), ratio=3:12026.06 | 0.5717 | — | 0.2687 | 0.4357 | 0.54 | 0.8885 | 0.552 | |
| Finetune (5:1)Backbone=Skywork-V2-Qwen3 (0.6B), ratio=5:12026.06 | 0.57 | — | 0.2532 | 0.4386 | 0.5394 | 0.885 | 0.5541 | |
| BaselineBackbone=Skywork-V2-Qwen3 (0.6B)2026.06 | 0.569 | — | 0.2636 | 0.4415 | 0.5432 | 0.8897 | 0.5117 | |
| HARVEBackbone=GRM-Llama-3.2 (3B), alpha=1.252026.06 | 0.5659 | — | 0.6434 | 0.4708 | 0.2886 | 0.9789 | 0.828 | |
| Finetune (5:1)Backbone=Skywork-Reward-Llama-3.1 (8B), ratio=5:12026.06 | 0.5479 | — | 0.3359 | 0.326 | 0.3667 | 0.9836 | 0.8662 | |
| Finetune (3:1)Backbone=Skywork-Reward-Llama-3.1 (8B), ratio=3:12026.06 | 0.5471 | — | 0.3333 | 0.326 | 0.3661 | 0.9836 | 0.8641 | |
| BaselineBackbone=Skywork-Reward-Llama-3.1 (8B)2026.06 | 0.5468 | — | 0.3256 | 0.3275 | 0.368 | 0.9836 | 0.8599 | |
| HARVEBackbone=FsfairX-LLaMA3-RM (8B), alpha=1.752026.06 | 0.5438 | — | 0.3127 | 0.4386 | 0.4058 | 0.9613 | 0.5966 | |
| Finetune (5:1)Backbone=GRM-Llama-3.2 (3B), ratio=5:12026.06 | 0.5318 | — | 0.3282 | 0.2646 | 0.3623 | 0.9742 | 0.8577 | |
| Finetune (3:1)Backbone=GRM-Llama-3.2 (3B), ratio=3:12026.06 | 0.5315 | — | 0.3282 | 0.2646 | 0.3617 | 0.9742 | 0.8577 | |
| BaselineBackbone=GRM-Llama-3.2 (3B)2026.06 | 0.5303 | — | 0.3178 | 0.2617 | 0.3623 | 0.973 | 0.8599 | |
| Finetune (5:1)Backbone=FsfairX-LLaMA3-RM (8B), ratio=5:12026.06 | 0.5247 | — | 0.2455 | 0.3933 | 0.4077 | 0.9519 | 0.5669 | |
| Finetune (3:1)Backbone=FsfairX-LLaMA3-RM (8B), ratio=3:12026.06 | 0.5094 | — | 0.2171 | 0.3713 | 0.3963 | 0.9296 | 0.5711 | |
| BaselineBackbone=FsfairX-LLaMA3-RM (8B)2026.06 | 0.5046 | — | 0.2067 | 0.3713 | 0.3919 | 0.919 | 0.5732 | |
| Finetune (5:1)Backbone=RM-Mistral (7B), ratio=5:12026.06 | 0.4788 | — | 0.2455 | 0.2661 | 0.3314 | 0.9519 | 0.62 | |
| Finetune (3:1)Backbone=RM-Mistral (7B), ratio=3:12026.06 | 0.4574 | — | 0.1912 | 0.2646 | 0.3075 | 0.9331 | 0.6008 | |
| BaselineBackbone=RM-Mistral (7B)2026.06 | 0.4441 | — | 0.1654 | 0.2529 | 0.3088 | 0.8862 | 0.6072 | |
| ArmoRM-Llama3-8B-v0.1Backbone=Llama3-8B2026.02 | — | 0.558 | — | — | — | — | — | |
| Athene-RM-8BSize=8B2026.02 | — | 0.514 | — | — | — | — | — | |
| INF-ORM-Llama3.1-70BBackbone=Llama3.1-70B2026.02 | — | 0.54 | — | — | — | — | — | |
| Internlm2-20b-rewardSize=20b2026.02 | — | 0.628 | — | — | — | — | — | |
| Llama-3-OffsetBias-RM-8BSize=8B2026.02 | — | 0.569 | — | — | — | — | — | |
| Llama-3.1-Nemotron-70BSize=70B2026.02 | — | 0.478 | — | — | — | — | — | |
| Skywork-Reward-Gemma-2-27B-v0.2Backbone=Gemma-2-27B2026.02 | — | 0.421 | — | — | — | — | — | |
| Skywork-Reward-Llama-3.1-8B-v0.2Backbone=Llama-3.1-8B2026.02 | — | 0.493 | — | — | — | — | — | |
| WILDREWARD-4BSize=4B2026.02 | — | 0.686 | — | — | — | — | — | |
| WILDREWARD-8BSize=8B2026.02 | — | 0.697 | — | — | — | — | — |