Reward Modeling Evaluation on RM-Bench
85.3Chat ScoreFlexible Principles ScalarRM
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Flexible Principles ScalarRMRM Type (Scalar vs Generative)=Scalar, Latency (seconds/task)=<0.1 second/task2025.09 | 85.3 | 81.9 | 70.4 | 96.9 | 85.5 | 84.9 | 80.5 | 83.6 | |
| Flexible Principles GenRMRM Type (Scalar vs Generative)=Generative, Latency (seconds/task)=>10 seconds/task2025.09 | 80.4 | 92 | 77 | 95.5 | 88.9 | 86.4 | 83.4 | 86.2 | |
| DSV4 + Eval-SkillJudge Backbone=DeepSeek-V4-Flash, Rubric Model=w/o2026.06 | 79.61 | 95.6 | 79.95 | 96.12 | — | — | — | 87.82 | |
| DSV4 + Naive SkillJudge Backbone=DeepSeek-V4-Flash, Rubric Model=w/o2026.06 | 78.06 | 91.66 | 83.33 | 96.68 | — | — | — | 87.43 | |
| Rubric-ARM-8B + Eval-SkillJudge Backbone=Qwen3-8B and Variants, Rubric Model=w/o2026.06 | 77.78 | 79.07 | 62.41 | 91.1 | — | — | — | 77.59 | |
| DSV4Judge Backbone=DeepSeek-V4-Flash, Rubric Model=DSV42026.06 | 76.79 | 79.69 | 76.48 | 90 | — | — | — | 80.74 | |
| RewardAnything-8B-v1RM Type (Scalar vs Generative)=Generative, Latency (seconds/task)=>10 seconds/task2025.09 | 76.7 | 90.3 | 75.2 | 90.2 | 85.6 | 82.2 | 81.5 | 83.1 | |
| R3-QWEN3-14B-LORA-4KRM Type (Scalar vs Generative)=Generative, Latency (seconds/task)=>10 seconds/task2025.09 | 76.5 | 92.4 | 78.7 | 91.9 | 91.4 | 86.2 | 77.1 | 84.9 | |
| Qwen-Instruct-32B-OursReward Model Type=ReasonRM2026.04 | 75.6 | 80 | 66.5 | 94.2 | 86 | 80.8 | 70.7 | 79.2 | |
| Llama-3.3-Nemotron-70B-RewardRM Type (Scalar vs Generative)=Scalar, Latency (seconds/task)=<0.1 second/task2025.09 | 75.4 | 84.5 | 69.3 | 90.4 | 92.1 | 84.1 | 63.5 | 79.9 | |
| Qwen-Instruct-14B-OursReward Model Type=ReasonRM2026.04 | 75.3 | 75.6 | 60.9 | 93.3 | 82.9 | 77.7 | 68.5 | 76.4 | |
| RM-R1(Qwen2.5-32B)Training=GRPO, Data=RM-R12026.05 | 75.27 | 81.39 | 64.81 | 93.67 | 85.69 | 81.01 | 69.65 | 78.79 | |
| RM-R1(Qwen2.5-14B)Training=GRPO, Data=RM-R12026.05 | 75.1 | 76.68 | 60.18 | 85.46 | 81.17 | 75.23 | 66.67 | 74.36 | |
| DSV4Judge Backbone=DeepSeek-V4-Flash, Rubric Model=w/o2026.06 | 74.96 | 78.3 | 72.05 | 92.6 | — | — | — | 79.48 | |
| Qwen3-8B + Eval-SkillJudge Backbone=Qwen3-8B and Variants, Rubric Model=w/o2026.06 | 74.4 | 82.36 | 59.98 | 92.6 | — | — | — | 77.34 | |
| RM-R1-DeepSeek-Distilled-Qwen-32BRM Type (Scalar vs Generative)=Generative, Latency (seconds/task)=>10 seconds/task2025.09 | 74.2 | 91.8 | 74.1 | 95.4 | 89.5 | 85.4 | 76.7 | 83.9 | |
| SKYWORK-RM (w/ calibration)Mismatch Degree=0.639, Calibration setting=w/ calibration2025.04 | 73.9 | 62.4 | 53.9 | 95.8 | 89.4 | 75.8 | 49.3 | 71.5 | |
| Llama-3.3-Nemotron-Super-49B-GenRMRM Type (Scalar vs Generative)=Generative, Latency (seconds/task)=>10 seconds/task2025.09 | 73.7 | 91.4 | 75 | 90.6 | 91.2 | 85.7 | 71.2 | 82.7 | |
| Bradley-TerryRM Type (Scalar vs Generative)=Scalar, Latency (seconds/task)=<0.1 second/task2025.09 | 73.6 | 82.7 | 66.1 | 91.4 | 89.2 | 82 | 64.2 | 78.5 | |
| SteerRMModel=URM2026.03 | 72.9 | 62 | 54.6 | 93.6 | 78.3 | 73 | 61 | 70.8 | |
| SteerRMModel=Skywork2026.03 | 72.5 | 62.5 | 56 | 95 | 83.9 | 74.1 | 56.5 | 71.5 | |
| BaselineModel=URM2026.03 | 72.2 | 61.6 | 53.4 | 94.7 | 83.9 | 73.7 | 53.7 | 70.5 | |
| DynaCFBackbone=Qwen3-8B2026.06 | 72.1 | 71 | 62.6 | 91.3 | 90.4 | 78 | 54.4 | 74.2 | |
| SteerRMModel=Base-RM2026.03 | 71.3 | 59.3 | 56.5 | 89.6 | 81.3 | 71.8 | 54.4 | 69.2 | |
| Llama-3-OffsetBias-RM-8BReward Model Type=Scalar RM2026.04 | 71.3 | 61.9 | 53.2 | 89.6 | 84.6 | 72.2 | 50.2 | 69 | |
| Nemotron-340B-RewardReward Model Type=Scalar RM2026.04 | 71.2 | 59.8 | 59.4 | 87.5 | 81 | 71.4 | 56.1 | 69.5 | |
| URM-LLaMa-3.1-8BReward Model Type=Scalar RM2026.04 | 71.2 | 61.8 | 54.1 | 93.1 | 84 | 73.2 | 53 | 70 | |
| URM-Llama-3.1-8BBackbone=Llama-3.1-8B2026.06 | 71.2 | 61.8 | 54.1 | 93.1 | 84 | 73.2 | 53 | 70 | |
| DynaCFBackbone=Qwen3-4B2026.06 | 71.2 | 69.2 | 63.4 | 88.7 | 91.1 | 78.2 | 50 | 73.1 | |
| Rubric-ARM-8BJudge Backbone=Qwen3-8B and Variants, Rubric Model=Rubric-ARM-8B2026.06 | 71.17 | 70.73 | 63.98 | 68.33 | — | — | — | 69.06 | |
| RRM-32BJudge Backbone=Other Post-Trained RMs, Rubric Model=w/o2026.06 | 71.03 | 88.19 | 71.44 | 91.82 | — | — | — | 80.62 | |
| BaselineModel=Base-RM2026.03 | 71 | 59.6 | 58.3 | 90.8 | 85.5 | 73.3 | 51 | 69.9 | |
| Qwen3-8B + Naive SkillJudge Backbone=Qwen3-8B and Variants, Rubric Model=w/o2026.06 | 70.89 | 72.05 | 57.81 | 90.52 | — | — | — | 72.82 | |
| Llama-3.1-Nemotron-70B-RewardRM Type (Scalar vs Generative)=Scalar, Latency (seconds/task)=<0.1 second/task2025.09 | 70.7 | 64.3 | 57.4 | 90.3 | 92.5 | 76.4 | 43.1 | 70.7 | |
| RM-R1(Qwen3-4B)Training=GRPO, Data=RM-R12026.05 | 70.63 | 72.19 | 57.21 | 92.28 | 81.44 | 73.99 | 63.75 | 73.06 | |
| BaselineModel=Skywork2026.03 | 69.8 | 60.6 | 54.5 | 96.5 | 88.9 | 74.9 | 47.3 | 70.3 | |
| Skywork-Reward-Llama-3.1-8BReward Model Type=Scalar RM2026.04 | 69.5 | 60.6 | 54.5 | 95.7 | 89 | 74.7 | 46.6 | 70.1 | |
| Skywork-Reward-Gemma-2-27BBackbone=Gemma-2-27B2026.06 | 69.5 | 54.7 | 53.2 | 91.9 | 78 | 69.2 | 54.9 | 67.3 | |
| Skywork-Reward-Llama-3.1-8BBackbone=Llama-3.1-8B2026.06 | 69.5 | 60.6 | 54.5 | 95.7 | 89 | 74.7 | 46.6 | 70.1 | |
| SKYWORK-RM (w/o calibration)Mismatch Degree=0.639, Calibration setting=w/o calibration2025.04 | 68.9 | 61.9 | 53.1 | 95.9 | 88.5 | 73.8 | 47.6 | 70 | |
| SKYWORK-RMMismatch Degree=0.639, Calibration setting=Original2025.04 | 68.7 | 62 | 52.8 | 95.9 | 88.4 | 73.7 | 47.5 | 69.9 | |
| tulu-v2.5-dpo-13b-hh-rlhf-60kReward Model Type=GenRM2026.04 | 68.4 | 51.1 | 52.3 | 76.5 | 53.6 | 63 | 69.6 | 62.1 | |
| Bradley–TerryBackbone=Qwen3-8B2026.06 | 68.1 | 69.8 | 60.8 | 90.2 | 90.3 | 76.6 | 49.8 | 72.2 | |
| Qwen3-4BJudge Backbone=Qwen3-4B and Variants, Rubric Model=w/o2026.06 | 67.93 | 70.14 | 54.17 | 85.24 | — | — | — | 69.37 | |
| SteerRMModel=Tulu2026.03 | 67.7 | 60.4 | 60.8 | 83.4 | 86.4 | 73.6 | 44.2 | 68.1 | |
| Qwen3-4B + Eval-SkillJudge Backbone=Qwen3-4B and Variants, Rubric Model=w/o2026.06 | 67.65 | 79.67 | 62.84 | 87.88 | — | — | — | 74.51 | |
| BaselineModel=QRM2026.03 | 67.4 | 63.2 | 52 | 95.6 | 87.8 | 73.4 | 47.5 | 69.5 | |
| RM-R1(Qwen2.5-7B)Training=GRPO, Data=RM-R12026.05 | 67.26 | 69.41 | 55.26 | 92.18 | 80.96 | 71.87 | 60.26 | 71.03 | |
| stablelm-2-12b-chatReward Model Type=GenRM2026.04 | 67.2 | 54.9 | 51.6 | 65.2 | 69.1 | 63.5 | 46.6 | 59.7 | |
| SteerRMModel=Ins-RM2026.03 | 67.1 | 65.1 | 59.9 | 91.1 | 87.1 | 75.1 | 50.1 | 70.8 | |
| RM-R1-DS-Qwen-7BJudge Backbone=Other Post-Trained RMs, Rubric Model=w/o2026.06 | 67.09 | 81.59 | 54.86 | 80.03 | — | — | — | 70.89 | |
| Qwen3-8BJudge Backbone=Qwen3-8B and Variants, Rubric Model=w/o2026.06 | 66.95 | 70.4 | 59.98 | 90.22 | — | — | — | 71.89 | |
| Qwen-Instruct-7B-OursReward Model Type=ReasonRM2026.04 | 66.9 | 66.8 | 54.4 | 92.9 | 79.5 | 71.3 | 59.5 | 70.1 | |
| BaselineModel=Ins-RM2026.03 | 66.8 | 65 | 57 | 91.6 | 90.7 | 75.8 | 43.8 | 70.1 | |
| tulu-v2.5-dpo-13b-stackexchange-60kReward Model Type=GenRM2026.04 | 66.4 | 49.9 | 54.2 | 69 | 79.5 | 63 | 37.2 | 59.9 | |
| tulu-2-dpo-13bReward Model Type=GenRM2026.04 | 66.4 | 51.4 | 51.8 | 85.4 | 86.9 | 66.7 | 37.7 | 63.8 | |
| GRM-RM (w/ calibration)Mismatch Degree=0.508, Calibration setting=w/ calibration2025.04 | 66.2 | 62.6 | 58 | 89.3 | 84.9 | 73.9 | 48.3 | 69 | |
| Qwen3-4B + Naive SkillJudge Backbone=Qwen3-4B and Variants, Rubric Model=w/o2026.06 | 65.68 | 72.57 | 56.86 | 88.53 | — | — | — | 70.91 | |
| SteerRMModel=QRM2026.03 | 65.5 | 62.9 | 52.8 | 95.2 | 81.1 | 71.2 | 54.9 | 69.1 | |
| Bradley–TerryBackbone=Qwen3-4B2026.06 | 65.5 | 67.3 | 62.6 | 80.5 | 91.3 | 74.4 | 41.3 | 69 | |
| tulu-v2.5-dpo-13b-chatbot-arena-2023Reward Model Type=GenRM2026.04 | 64.9 | 52.3 | 50.5 | 62.3 | 82.8 | 60.2 | 29.5 | 57.5 | |
| BaselineModel=Tulu2026.03 | 64.8 | 60.2 | 57.7 | 83.5 | 91.9 | 73.8 | 33.9 | 66.5 | |
| Rubric-RM-8BJudge Backbone=Qwen3-8B and Variants, Rubric Model=Rubric-RM-8B2026.06 | 64.7 | 69.85 | 60.5 | 54.19 | — | — | — | 62.31 | |
| FSFAIRX-RM (w/ calibration)Mismatch Degree=0.554, Calibration setting=w/ calibration2025.04 | 64.5 | 63.3 | 56 | 90 | 87.5 | 72.6 | 45.4 | 68.5 | |
| Qwen3-4BJudge Backbone=Qwen3-4B and Variants, Rubric Model=Rubric-RM-4B2026.06 | 63.71 | 66.23 | 55.47 | 69.01 | — | — | — | 63.61 | |
| Qwen3-8BJudge Backbone=Qwen3-8B and Variants, Rubric Model=Rubric-ARM-8B2026.06 | 63.63 | 67.68 | 57.29 | 77.48 | — | — | — | 66.52 | |
| GRM-RMMismatch Degree=0.508, Calibration setting=Original2025.04 | 63.6 | 62 | 56.9 | 89.1 | 82.2 | 71.8 | 49.6 | 67.9 | |
| GRM-RM (w/o calibration)Mismatch Degree=0.508, Calibration setting=w/o calibration2025.04 | 63.6 | 62.4 | 58.3 | 89.5 | 82.9 | 72.7 | 49.8 | 68.4 | |
| Qwen3-8BJudge Backbone=Qwen3-8B and Variants, Rubric Model=Rubric-RM-8B2026.06 | 63.43 | 67.91 | 56.94 | 75.53 | — | — | — | 65.95 | |
| MISTRAL-RM (w/ calibration)Mismatch Degree=0.528, Calibration setting=w/ calibration2025.04 | 63.2 | 57 | 52.4 | 88.3 | 89.5 | 69.8 | 36.3 | 65.2 | |
| FSFAIRX-RM (w/o calibration)Mismatch Degree=0.554, Calibration setting=w/o calibration2025.04 | 63.1 | 63.4 | 53.6 | 90.4 | 85.4 | 71.6 | 45.8 | 67.6 | |
| internlm2-20b-rewardReward Model Type=Scalar RM2026.04 | 63.1 | 66.8 | 56.7 | 86.5 | 82.6 | 71.6 | 50.7 | 68.3 | |
| Qwen3-8BJudge Backbone=Qwen3-8B and Variants, Rubric Model=DSV42026.06 | 62.73 | 71.72 | 58.33 | 89.41 | — | — | — | 70.55 | |
| GRM-llama3-8B-sftregReward Model Type=Scalar RM2026.04 | 62.7 | 62.5 | 57.8 | 90 | 83.5 | 72.7 | 48.6 | 68.2 | |
| FSFAIRX-RMMismatch Degree=0.554, Calibration setting=Original2025.04 | 62.5 | 63.2 | 54.6 | 90.4 | 86.5 | 71.6 | 44.9 | 67.7 | |
| RRM-7BJudge Backbone=Other Post-Trained RMs, Rubric Model=w/o2026.06 | 62.31 | 75.24 | 53.73 | 80.06 | — | — | — | 67.84 | |
| Qwen3-4BJudge Backbone=Qwen3-4B and Variants, Rubric Model=DSV42026.06 | 62.17 | 71.38 | 57.81 | 87.68 | — | — | — | 69.76 | |
| RM-R1-Qwen2.5-7BJudge Backbone=Other Post-Trained RMs, Rubric Model=w/o2026.06 | 61.74 | 58.64 | 53.3 | 80.51 | — | — | — | 63.55 | |
| internlm2-7b-rewardReward Model Type=Scalar RM2026.04 | 61.7 | 71.4 | 49.7 | 85.5 | 85.4 | 70.7 | 45.1 | 67.1 | |
| MISTRAL-RM (w/o calibration)Mismatch Degree=0.528, Calibration setting=w/o calibration2025.04 | 61.4 | 57.4 | 53 | 88.9 | 86.7 | 68.8 | 40 | 65.2 | |
| MISTRAL-RMMismatch Degree=0.528, Calibration setting=Original2025.04 | 60.8 | 56.6 | 52.6 | 88.7 | 88.3 | 68.2 | 37.5 | 64.7 | |
| BTRM-RM (w/ calibration)Mismatch Degree=0.162, Calibration setting=w/ calibration2025.04 | 60.2 | 60.5 | 53.8 | 89.6 | 92.1 | 71.1 | 34.8 | 66 | |
| BTRM-RMMismatch Degree=0.162, Calibration setting=Original2025.04 | 60 | 61.3 | 53.8 | 89.9 | 90.7 | 71 | 37.1 | 66.3 | |
| Eurus-RM-7bReward Model Type=Scalar RM2026.04 | 59.9 | 60.2 | 56.9 | 86.5 | 87.2 | 70.2 | 40.2 | 65.9 | |
| Nous-Hermes-2-Mistral-7B-DPOReward Model Type=GenRM2026.04 | 58.8 | 55.6 | 51.3 | 73.9 | 69.5 | 61.1 | 49.1 | 59.9 | |
| BTRM-RM (w/o calibration)Mismatch Degree=0.162, Calibration setting=w/o calibration2025.04 | 58.5 | 61.5 | 54.1 | 89.1 | 91.3 | 70.8 | 35.3 | 65.8 | |
| tulu-v2.5-70b-preference-mix-rmReward Model Type=Scalar RM2026.04 | 58.2 | 51.4 | 55.5 | 87.1 | 72.8 | 65.6 | 50.7 | 63 | |
| ActivationModel=URM2026.03 | 58.1 | 49.7 | 47.7 | 51.8 | 20.3 | 54.2 | 80.9 | 51.8 | |
| ActivationModel=Skywork2026.03 | 58 | 50.3 | 50 | 50.1 | 20.8 | 54 | 81.5 | 52.1 | |
| RM-Mistral-7BReward Model Type=Scalar RM2026.04 | 57.4 | 57 | 52.7 | 87.2 | 88.6 | 67.1 | 34.9 | 63.5 | |
| Mistral-7B-instruct-Unified-FeedbackReward Model Type=Scalar RM2026.04 | 56.5 | 58 | 51.7 | 86.8 | 87.1 | 67.3 | 35.3 | 63.2 | |
| steerlm-70bReward Model Type=Scalar RM2026.04 | 56.4 | 53 | 49.3 | 51.2 | 48.3 | 54.9 | 54.3 | 52.5 | |
| tulu-v2.5-dpo-13b-nectar-60kReward Model Type=GenRM2026.04 | 56.3 | 52.4 | 52.6 | 73.8 | 86.7 | 64.3 | 25.4 | 58.8 | |
| ActivationModel=Ins-RM2026.03 | 55.8 | 51.5 | 46.2 | 84.7 | 65.1 | 60.9 | 52.6 | 59.5 | |
| RM-R1(Qwen3-0.6B)Training=GRPO, Data=RM-R12026.05 | 55.04 | 61.73 | 53.65 | 63.24 | 73.75 | 58.36 | 43.14 | 58.42 | |
| ActivationModel=Tulu2026.03 | 52.5 | 50.1 | 49.1 | 44.3 | 53.9 | 48.6 | 44.4 | 49 | |
| Rubric-RM-4BJudge Backbone=Qwen3-4B and Variants, Rubric Model=Rubric-RM-4B2026.06 | 48.1 | 48.59 | 51.82 | 63.83 | — | — | — | 53.09 | |
| ActivationModel=QRM2026.03 | 47.9 | 49.4 | 47.2 | 46 | 46.8 | 46.4 | 49.6 | 47.6 | |
| ActivationModel=Base-RM2026.03 | 44.4 | 49.4 | 47.8 | 55.3 | 43.1 | 48.7 | 55.7 | 49.2 |