Reward Modeling on RewardBench
96.7Safety ScoreOracle Ensemble
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Oracle EnsembleBackbone=GRM-Gemma2-2B, Type=Seq + Oracle, Weighting scheme=best weight oracle2024.12 | 96.7 | 93.1 | 98.3 | 83.4 | 94.9 | — | |
| Oracle EnsembleBackbone=Skywork–Llama-3.1-8B, Type=Seq + Oracle, Weighting scheme=best weight oracle2024.12 | 96.5 | 97.2 | 99.2 | 93 | 98.8 | — | |
| Skywork2026.06 | 95.7 | — | 99.4 | 90.4 | 99.6 | — | |
| Eval-Skill (full)Judge Backbone=Qwen2.5-32B-Instruct2026.06 | 94.11 | 93.4 | 95.61 | 84.55 | 99.32 | — | |
| Eval-Skill (adaptive)Judge Backbone=Qwen2.5-32B-Instruct2026.06 | 94.11 | 93.4 | 95.61 | 84.55 | 99.32 | — | |
| Eval-Skill (workflow-only)Judge Backbone=Qwen2.5-32B-Instruct2026.06 | 93.96 | 92.39 | 94.06 | 82.12 | 99.42 | — | |
| INF-ORM-Llama3.1-70BLLM=Llama3.1-70B2025.01 | 93.6 | 95.1 | 96.6 | 91 | 99.1 | — | |
| INF-ORM-Llama3.1-70BType=Seq2024.12 | 93.6 | 95.1 | 96.6 | 91 | 99.1 | — | |
| HyReBackbone=Skywork–Llama-3.1-8B, Type=Seq + HyRe, Adaptation set size (N)=102024.12 | 93.5 | 95 | 95.9 | 89.3 | 97.9 | — | |
| HyReBackbone=Skywork–Llama-3.1-8B, Type=Seq + HyRe, Adaptation set size (N)=52024.12 | 93.2 | 94.7 | 95.5 | 88.6 | 97.8 | — | |
| HRCBase Model=Llama-3.1-8B-Instruct, DIM=4 + 12026.05 | 93.11 | 91.99 | 94.13 | 85.53 | 95.18 | — | |
| DSV4 + Naive SkillJudge Backbone=DeepSeek-V4-Flash, Rubric Model=w/o2026.06 | 93.02 | 92.45 | 94.19 | 84.55 | 98.02 | — | |
| Uniform EnsembleBackbone=Skywork–Llama-3.1-8B, Type=Seq, Weighting scheme=uniform2024.12 | 93 | 94 | 95 | 87.2 | 96.8 | — | |
| HyReBackbone=Skywork–Llama-3.1-8B, Type=Seq + HyRe, Adaptation set size (N)=12024.12 | 93 | 94.3 | 95.2 | 87.8 | 97.5 | — | |
| BTBase Model=Llama-3.1-8B-Instruct, DIM=12026.05 | 92.97 | 90.46 | 89.11 | 84.86 | 94.9 | — | |
| Llama 3 70B RM (w. HelpSteer2)Source of Model/Training Data=Trained with Data allowing Permissive Use2024.06 | 92.8 | 88.8 | 91.3 | 80.3 | 90.7 | — | |
| SteerLM-RM 70BRM Type=Scalar RMs2026.04 | 92.8 | — | 91.3 | 80.3 | 90.6 | 88.8 | |
| Cohere May 2024Source of Model/Training Data=Proprietary Models2024.06 | 92.7 | 89.5 | 96.4 | 71.3 | 97.7 | — | |
| Skywork–Llama-3.1-8BType=Seq2024.12 | 92.7 | 94 | 94.7 | 88.6 | 96.7 | — | |
| HRCBase Model=Llama-3.1-8B-Instruct, DIM=2 + 12026.05 | 92.7 | 91.84 | 93.58 | 85.96 | 95.11 | — | |
| DSV4 + Eval-SkillJudge Backbone=DeepSeek-V4-Flash, Rubric Model=w/o2026.06 | 92.55 | 93.75 | 95.87 | 87.55 | 99.02 | — | |
| HyReBackbone=GRM-Gemma2-2B, Type=Seq + HyRe, Adaptation set size (N)=102024.12 | 92.4 | 89.7 | 96.4 | 74.7 | 93.5 | — | |
| Cohere-0514RM Type=Scalar RMs2026.04 | 92.3 | — | 96.4 | 71.3 | 97.7 | 89.4 | |
| ArmoRM-Llama 3 8BSource of Model/Training Data=Trained with GPT-4 Generated Data2024.06 | 92.2 | 90.8 | 96.9 | 76.8 | 97.3 | — | |
| GRM-Gemma2-2BType=Seq2024.12 | 92.2 | 88.4 | 93 | 77.2 | 91.2 | — | |
| GPMBase Model=Llama-3.1-8B-Instruct, DIM=42026.05 | 92.16 | 91.08 | 92.46 | 83.91 | 95.81 | — | |
| Qwen-Instruct-32B-OursRM Type=ReasonRMs2026.04 | 92 | — | 95.4 | 83.3 | 95.4 | 91.5 | |
| DirectJudge Backbone=Qwen2.5-32B-Instruct2026.06 | 91.98 | 91.71 | 93.02 | 83.15 | 98.67 | — | |
| Skywork-Gemma-2-27B†‡Model Category=Scalar Reward Models2025.10 | 91.9 | 93.8 | 95.8 | 91.4 | 96.1 | — | |
| Skywork-Reward-Gemma-2-27BBackbone=Gemma-2-27B2026.06 | 91.9 | 93.8 | — | — | — | — | |
| GPMBase Model=Llama-3.1-8B-Instruct, DIM=22026.05 | 91.76 | 91.14 | 92.74 | 85.3 | 94.76 | — | |
| Nemotron-4 340B RM (w. HelpSteer2)Source of Model/Training Data=Proprietary Models2024.06 | 91.5 | 92 | 95.8 | 87.1 | 93.7 | — | |
| OSS2026.06 | 91.4 | — | 96.1 | 88.8 | 98.3 | — | |
| Eval-Skill (full)Judge Backbone=Qwen2-72B-Instruct2026.06 | 91.2 | 90.5 | 93.8 | 78.65 | 98.35 | — | |
| Eval-Skill (adaptive)Judge Backbone=Qwen2-72B-Instruct2026.06 | 91.2 | 91.79 | 93.8 | 83.8 | 98.35 | — | |
| UltraMedRM-8BParameters=8B2024.06 | 91.19 | 85.53 | 97.21 | 67.11 | 86.62 | — | |
| URM-Llama-3.1-8BBackbone=Llama-3.1-8B2026.06 | 91.1 | 92.9 | — | — | — | — | |
| RM-R1(Qwen2.5-32B)Training=GRPO, Data=RM-R12026.05 | 90.95 | — | 93.29 | 82.02 | 94.72 | 90.25 | |
| Skyword-Reward-Llama3.1-8BLLM=Llama3.1-8B2025.01 | 90.8 | 92.5 | 95.8 | 87.3 | 96.2 | — | |
| Skywork-Llama-3.1-8B†‡Model Category=Scalar Reward Models2025.10 | 90.8 | 92.5 | 95.8 | 87.3 | 96.2 | — | |
| Skywork-Reward-Llama-3.1-8BBackbone=Llama-3.1-8B2026.06 | 90.8 | 92.5 | — | — | — | — | |
| Eval-Skill (full)Judge Backbone=Gemma-2-9B-IT2026.06 | 90.73 | 90.8 | 95.61 | 79.4 | 97.44 | — | |
| Eval-Skill (adaptive)Judge Backbone=Gemma-2-9B-IT2026.06 | 90.73 | 90.7 | 95.22 | 79.4 | 97.44 | — | |
| RM-DistillerStudent=Qwen2.5-3B-Instruct, Teacher=GPT-4o, Scenario=Labeled Scene2026.01 | 90.7 | 83.6 | 92.7 | 64.3 | 86.9 | — | |
| Random (Uniform)Base Model=Llama3-8B-RM, n=152025.02 | 90.3 | 85.7 | 98.5 | 62.3 | 91.5 | — | |
| Cohere March 2024Source of Model/Training Data=Proprietary Models2024.06 | 90.3 | 87.1 | 94.7 | 65.1 | 98.2 | — | |
| HyReBackbone=GRM-Gemma2-2B, Type=Seq + HyRe, Adaptation set size (N)=52024.12 | 90.3 | 88.5 | 95 | 72.5 | 93.1 | — | |
| BT (baseline)Base Model=Llama-3.2-3B-Instruct, Training Dataset=Skywork-Reward-Preference-80K-v0.22025.12 | 89.86 | 80.31 | 86.03 | 78.29 | 67.05 | — | |
| Vanilla RMModel Scale=Gemma-9B-it, k=6.4e42026.04 | 89.73 | 83.09 | 96.37 | 63.37 | 82.88 | 83.22 | |
| RLHFlow-Llama 3 8BSource of Model/Training Data=Trained with GPT-4 Generated Data2024.06 | 89.7 | 87.1 | 98.3 | 65.8 | 94.7 | — | |
| PairRM-LLaMA3-8BBackbone=LLaMA3-8B, Type=Pairwise2024.06 | 89.7 | 87.13 | 98.3 | 65.8 | 94.7 | — | |
| Self-taught-evaluator-llama3.1-70BRM Type=GenRMs2026.04 | 89.6 | — | 96.9 | 85.1 | 88.4 | 90 | |
| Shared-BaseBase Model=Llama3-8B-RM, n=152025.02 | 89.5 | 87.3 | 98.6 | 68.4 | 92.7 | — | |
| InternLM2-20B-RewardLLM=InternLM2-20B2025.01 | 89.5 | 90.2 | 98.9 | 76.5 | 95.8 | — | |
| Eval-Skill (workflow-only)Judge Backbone=Gemma-2-9B-IT2026.06 | 89.48 | 89.87 | 95.22 | 77.34 | 97.42 | — | |
| GRMSize=8B, Backbone=llama3-8b-instruct2024.06 | 89.4 | 87 | 98.6 | 67.8 | 92.3 | — | |
| Q3-8B + Eval-SkillJudge Backbone=Qwen3-8B and Variants, Rubric Model=w/o2026.06 | 89.11 | 86.97 | 93.67 | 72.66 | 92.43 | — | |
| SKYWORK + ULTRAFEEDBACKTraining Data=SKYWORK + ULTRAFEEDBACK, Type=Annotated, Size=140k2024.10 | 89.1 | 85.1 | 94.7 | 72.7 | 83.9 | — | |
| Single HeadBase Model=Llama3-8B-RM, n=152025.02 | 89.1 | 87.1 | 98.9 | 68.4 | 92 | — | |
| Claude-3-Opus-02292024Source of Model/Training Data=Proprietary Models2024.06 | 89.1 | 80.7 | 94.7 | 60.3 | 78.7 | — | |
| Random (Gaussian)Base Model=Llama3-8B-RM, n=152025.02 | 88.9 | 86.8 | 98.2 | 66.3 | 93.6 | — | |
| NORMBTBase Model=Llama-3.2-3B-Instruct, Training Dataset=Skywork-Reward-Preference-80K-v0.22025.12 | 88.78 | 81.48 | 83.8 | 78.73 | 74.6 | — | |
| Rubric-ARM-8B + Eval-SkillJudge Backbone=Qwen3-8B and Variants, Rubric Model=w/o2026.06 | 88.59 | 88.43 | 95.09 | 80.24 | 89.8 | — | |
| DRMs(Ours)Base Model=Llama3-8B-RM, n=152025.02 | 88.5 | 88.5 | 98.6 | 75.5 | 91.4 | — | |
| CARPModel Scale=Gemma-9B-it, k=6.4e42026.04 | 88.24 | 85.42 | 94.69 | 68.86 | 89.87 | 86.83 | |
| Q3-4B + Eval-SkillJudge Backbone=Qwen3-4B and Variants, Rubric Model=w/o2026.06 | 88.23 | 85.52 | 93.67 | 70.79 | 89.4 | — | |
| SKYWORKTraining Data=SKYWORK (Liu et al., 2024), Type=Annotated, Size=80k2024.10 | 88.2 | 84.5 | 91.6 | 78.6 | 79.7 | — | |
| Starling RM Yi 34BSource of Model/Training Data=Trained with GPT-4 Generated Data2024.06 | 88.2 | 82.7 | 96.9 | 57.2 | 88.5 | — | |
| Starling-RM-34BSize=34B2024.06 | 88.2 | 82.7 | 96.9 | 57.2 | 88.5 | — | |
| DSV4Judge Backbone=DeepSeek-V4-Flash, Rubric Model=w/o2026.06 | 88.12 | 89.66 | 94.96 | 80.06 | 95.49 | — | |
| RM-R1(Qwen3-4B)Training=GRPO, Data=RM-R12026.05 | 88.11 | — | 91.62 | 75.87 | 86.52 | 86.13 | |
| LSAMStudent=Qwen2.5-3B-Instruct, Teacher=GPT-4o, Scenario=Labeled Scene2026.01 | 88.1 | 77 | 89.4 | 66.5 | 63.8 | — | |
| GPT-4 (Aug 2024)Type=Gen2024.12 | 88.1 | 86.7 | 96.1 | 76.1 | 86.6 | — | |
| Self-Rubrics OSSbaseline=self-generated rubrics, model=OSS2026.06 | 88.1 | — | 80.7 | 72.9 | 87 | — | |
| Q3-8B + Naive SkillJudge Backbone=Qwen3-8B and Variants, Rubric Model=w/o2026.06 | 88.02 | 83.51 | 93.02 | 67.98 | 85.01 | — | |
| Qwen3-14BType=LLM-as-a-Judge2026.01 | 88 | 83.9 | 96.4 | 69.2 | 82 | — | |
| DeepSeek-GRM-27BRM Type=ReasonRMs2026.04 | 88 | — | 94.1 | 78.3 | 83.8 | 86 | |
| RISE-Judge-Qwen2.5-7BType=Gen2024.12 | 88 | 88.2 | 92.2 | 76.5 | 96.1 | — | |
| Eval-Skill (workflow-only)Judge Backbone=Qwen2-72B-Instruct2026.06 | 87.92 | 91.44 | 95.61 | 83.8 | 98.42 | — | |
| Gemini-1.5-pro-0514RM Type=GenRMs2026.04 | 87.9 | — | 92.3 | 80.6 | 92 | 88.2 | |
| FsfairX-LLaMA3-RM-8BSize=8B, Backbone=LLaMA32024.06 | 87.8 | 84.7 | 99.4 | 65.1 | 86.4 | — | |
| IXC-2.5-RewardLLM=InternLM2-7B2025.01 | 87.8 | 88.6 | 90.8 | 83.8 | 90 | — | |
| GPT-4oType=LLM-as-a-Judge2026.01 | 87.8 | 82.4 | 96.1 | 70.7 | 75.1 | — | |
| PaTaRM Qwen3-14BModel Family=Qwen3-14B Family, Model Type=Generative RM (PaTaRM)2025.10 | 87.8 | 87.2 | 91.5 | 77.9 | 91.5 | — | |
| RM-R1(Qwen2.5-14B)Training=GRPO, Data=RM-R12026.05 | 87.7 | — | 93.57 | 79.38 | 91.4 | 88.01 | |
| BT-Qwen3-14B (Scalar)Model Family=Qwen3-14B Family, Model Type=Scalar RM (BT)2025.10 | 87.6 | 89.9 | 95.3 | 87.5 | 89.2 | — | |
| Q3-4B + Naive SkillJudge Backbone=Qwen3-4B and Variants, Rubric Model=w/o2026.06 | 87.55 | 78.9 | 92.12 | 52.06 | 83.86 | — | |
| DirectJudge Backbone=Qwen2-72B-Instruct2026.06 | 87.55 | 90.12 | 93.8 | 81.18 | 97.94 | — | |
| Gemini 1.5 Pro-0514Source of Model/Training Data=Proprietary Models2024.06 | 87.5 | 88.1 | 92.3 | 80.6 | 92 | — | |
| Uniform EnsembleBackbone=GRM-Gemma2-2B, Type=Seq, Weighting scheme=uniform2024.12 | 87.4 | 87.1 | 96.4 | 73.1 | 89.8 | — | |
| BT-Qwen3-8B (Scalar)Model Family=Qwen3-8B Family, Model Type=Scalar RM (BT)2025.10 | 87.4 | 86.3 | 96.4 | 79.6 | 82 | — | |
| Margin BTStudent=Qwen2.5-3B-Instruct, Teacher=GPT-4o, Scenario=Labeled Scene2026.01 | 87.3 | 78.6 | 88.6 | 70.8 | 67.8 | — | |
| DirectJudge Backbone=Gemma-2-9B-IT2026.06 | 87.29 | 85.03 | 95.61 | 70.69 | 86.51 | — | |
| GPT-4-0125-previewSource of Model/Training Data=Proprietary Models2024.06 | 87.2 | 85.9 | 95.3 | 74.3 | 86.9 | — | |
| gpt-4-0125-preview2024.06 | 87.2 | 85.9 | 95.3 | 74.3 | 86.9 | — | |
| InternLM2-7B-RewardLLM=InternLM2-7B2025.01 | 87.2 | 87.6 | 99.2 | 69.5 | 94.5 | — | |
| SteerLMStudent=Qwen2.5-3B-Instruct, Teacher=Qwen3-14B, Scenario=Labeled Scene2026.01 | 87.2 | 78.4 | 89.7 | 69.5 | 67.1 | — | |
| Qwen-Instruct-14B-OursRM Type=ReasonRMs2026.04 | 87.2 | — | 93.8 | 80.6 | 92.1 | 88.4 | |
| PaTaRM Qwen3-8BModel Family=Qwen3-8B Family, Model Type=Generative RM (PaTaRM)2025.10 | 87.2 | 84.3 | 87.7 | 74.3 | 87.8 | — | |
| GPT-4-0613Source of Model/Training Data=Proprietary Models2024.06 | 87.1 | 85.1 | 95.3 | 75.4 | 82.7 | — |