Reward Modeling on REWARDHACKBENCH (Tgt, Non-Tgt, Surface, Reasoning, Sycophantic, Off-Topic, Style)
92.42Target AdherenceHARVE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| HARVEBackbone=Skywork-V2-Llama-3.2 (3B), alpha=1.252026.06 | 92.42 | 86.52 | 86.36 | 93.57 | 96.67 | 82.74 | 83.42 | |
| HARVEBackbone=RM-Mistral (7B), alpha=2.502026.06 | 90.25 | 76.52 | 88.38 | 92.14 | 87.62 | 65.49 | 73.58 | |
| HARVEBackbone=InternLM2-reward (20B), alpha=1.602026.06 | 88.41 | 80.9 | 92.42 | 90.71 | 89.05 | 63.72 | 83.42 | |
| HARVEBackbone=FsfairX-LLaMA3-RM (8B), alpha=1.752026.06 | 86.96 | 76.58 | 80.81 | 94.29 | 86.19 | 62.39 | 76.68 | |
| HARVEBackbone=Skywork-V2-Qwen3 (0.6B), alpha=1.252026.06 | 85.5 | 77.51 | 80.81 | 82.14 | 92.38 | 69.03 | 75.65 | |
| Finetune (5:1)Backbone=RM-Mistral (7B), ratio=5:12026.06 | 84.48 | 78.55 | 94.95 | 82.14 | 93.33 | 57.52 | 76.17 | |
| HARVEBackbone=Skywork-Reward-Llama-3.1 (8B), alpha=1.102026.06 | 84.42 | 89.58 | 80.3 | 93.57 | 96.19 | 91.15 | 79.79 | |
| Finetune (3:1)Backbone=RM-Mistral (7B), ratio=3:12026.06 | 84.12 | 71.3 | 94.44 | 81.43 | 93.81 | 45.13 | 64.77 | |
| HARVEBackbone=GRM-Llama-3.2 (3B), alpha=1.252026.06 | 82.9 | 84.38 | 72.73 | 90.71 | 89.52 | 90.27 | 77.2 | |
| HARVEBackbone=InternLM2-reward (1.8B), alpha=1.102026.06 | 78.93 | 75.4 | 87.88 | 78.57 | 83.81 | 55.31 | 79.79 | |
| Finetune (5:1)Backbone=Skywork-V2-Llama-3.2 (3B), ratio=5:12026.06 | 76.9 | 86.09 | 80.3 | 89.29 | 84.29 | 81.42 | 83.94 | |
| Finetune (3:1)Backbone=Skywork-V2-Llama-3.2 (3B), ratio=3:12026.06 | 76.17 | 86.23 | 79.29 | 89.29 | 84.29 | 81.42 | 84.46 | |
| Finetune (5:1)Backbone=InternLM2-reward (20B), ratio=5:12026.06 | 75.36 | 83.07 | 84.85 | 83.57 | 83.81 | 66.81 | 88.08 | |
| BaselineBackbone=Skywork-V2-Llama-3.2 (3B)2026.06 | 75.09 | 85.51 | 78.28 | 89.29 | 81.9 | 81.42 | 83.94 | |
| Finetune (5:1)Backbone=Skywork-Reward-Llama-3.1 (8B), ratio=5:12026.06 | 73.55 | 92.19 | 77.78 | 85.71 | 92.38 | 91.59 | 85.49 | |
| Finetune (3:1)Backbone=InternLM2-reward (20B), ratio=3:12026.06 | 73.55 | 83.07 | 84.34 | 80.71 | 83.81 | 66.81 | 88.08 | |
| BaselineBackbone=InternLM2-reward (20B)2026.06 | 73.19 | 82.78 | 83.84 | 80.71 | 82.86 | 66.81 | 88.08 | |
| Finetune (3:1)Backbone=Skywork-Reward-Llama-3.1 (8B), ratio=3:12026.06 | 72.46 | 91.75 | 76.77 | 85.71 | 91.43 | 91.59 | 84.46 | |
| Finetune (5:1)Backbone=FsfairX-LLaMA3-RM (8B), ratio=5:12026.06 | 71.5 | 80.13 | 89.9 | 82.14 | 85.71 | 58.41 | 78.76 | |
| BaselineBackbone=RM-Mistral (7B)2026.06 | 67.87 | 72.32 | 83.84 | 79.29 | 78.57 | 47.35 | 71.5 | |
| Finetune (3:1)Backbone=FsfairX-LLaMA3-RM (8B), ratio=3:12026.06 | 67.63 | 77.89 | 85.86 | 80.71 | 83.81 | 55.75 | 76.17 | |
| BaselineBackbone=Skywork-Reward-Llama-3.1 (8B)2026.06 | 65.94 | 91.17 | 72.22 | 84.29 | 85.71 | 92.04 | 84.46 | |
| Finetune (5:1)Backbone=InternLM2-reward (1.8B), ratio=5:12026.06 | 62.5 | 75.25 | 90.91 | 68.57 | 67.14 | 53.98 | 79.27 | |
| Finetune (3:1)Backbone=InternLM2-reward (1.8B), ratio=3:12026.06 | 61.07 | 74.82 | 90.4 | 70 | 65.24 | 52.65 | 78.76 | |
| BaselineBackbone=InternLM2-reward (1.8B)2026.06 | 58.93 | 74.24 | 89.9 | 66.43 | 63.33 | 52.21 | 79.27 | |
| Finetune (3:1)Backbone=Skywork-V2-Qwen3 (0.6B), ratio=3:12026.06 | 58.36 | 74.93 | 60.61 | 75.71 | 85.71 | 56.64 | 75.65 | |
| Finetune (5:1)Backbone=Skywork-V2-Qwen3 (0.6B), ratio=5:12026.06 | 58.36 | 75.5 | 61.62 | 75 | 86.19 | 57.52 | 75.65 | |
| BaselineBackbone=FsfairX-LLaMA3-RM (8B)2026.06 | 57.49 | 75.66 | 81.31 | 78.57 | 74.29 | 53.54 | 75.65 | |
| Finetune (5:1)Backbone=GRM-Llama-3.2 (3B), ratio=5:12026.06 | 54.28 | 86.25 | 61.62 | 83.57 | 72.86 | 88.94 | 80.31 | |
| BaselineBackbone=Skywork-V2-Qwen3 (0.6B)2026.06 | 53.16 | 73.07 | 56.57 | 73.57 | 82.86 | 52.21 | 75.65 | |
| Finetune (3:1)Backbone=GRM-Llama-3.2 (3B), ratio=3:12026.06 | 50.56 | 85.96 | 59.09 | 85 | 68.57 | 88.94 | 80.31 | |
| BaselineBackbone=GRM-Llama-3.2 (3B)2026.06 | 46.47 | 84.81 | 55.05 | 83.57 | 64.76 | 88.94 | 79.79 |