Reward Modeling on PPE-Preference
79.8AccuracySkywork-Reward-V2-Llama-3.1-8B-40M
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Skywork-Reward-V2-Llama-3.1-8B-40MParadigm=Our Reward Models2025.07 | 79.8 | — | — | |
| Skywork-Reward-V2-Llama-3.1-8BParadigm=Our Reward Models2025.07 | 77.3 | — | — | |
| Skywork-Reward-V2-Qwen3-8BParadigm=Our Reward Models2025.07 | 70.6 | — | — | |
| Gemini-2.5-ProCategory=LLM-as-Judge, Tier=Pro2026.02 | 69.8 | 8.3 | 64 | |
| Skywork-Reward-V2-Qwen3-4BParadigm=Our Reward Models2025.07 | 69.5 | — | — | |
| Claude-Sonnet-4.5Category=LLM-as-Judge2026.02 | 69.2 | 14.5 | 59.1 | |
| Skywork-Reward-V2-Llama-3.2-3BParadigm=Our Reward Models2025.07 | 69.1 | — | — | |
| Claude-Sonnet-4.5-thinkingCategory=LLM-as-Judge, Mode=thinking2026.02 | 68.7 | 8.7 | 62.7 | |
| SAVEPolicy Model=Qwen3-4B-Instruct-25072026.05 | 68.6 | — | — | |
| Continual Offline Training RM2026.05 | 68.5 | — | — | |
| Skywork-Reward-V2-Llama-3.2-3B2026.05 | 68.4 | — | — | |
| HL-BTPolicy Model=Qwen2.5-3B-Instruct2026.05 | 68.4 | — | — | |
| HL-BTPolicy Model=Qwen3-4B-Instruct-25072026.05 | 68.3 | — | — | |
| SAVE (w/o Policy Model Optimization)Policy Model=Qwen2.5-3B-Instruct2026.05 | 67.9 | — | — | |
| SAVE (w/o Curriculum Mechanism)Policy Model=Qwen3-4B-Instruct-25072026.05 | 67.8 | — | — | |
| SAVE (w/o Policy Model Optimization)Policy Model=Qwen3-4B-Instruct-25072026.05 | 67.8 | — | — | |
| GPT-4oParadigm=LLM-as-a-Judge & Generative Reward Models2025.07 | 67.7 | — | — | |
| Skywork-Reward-V2-Qwen3-1.7BParadigm=Our Reward Models2025.07 | 67.6 | — | — | |
| SAVEPolicy Model=Qwen2.5-3B-Instruct2026.05 | 67.5 | — | — | |
| SAVE (w/o Curriculum Mechanism)Policy Model=Qwen2.5-3B-Instruct2026.05 | 67.5 | — | — | |
| Claude-3.5-SonnetParadigm=LLM-as-a-Judge & Generative Reward Models2025.07 | 67.3 | — | — | |
| DeepSeek-GRM-27BParadigm=LLM-as-a-Judge & Generative Reward Models, Integration=w/ MetaRM2025.07 | 67.2 | — | — | |
| J1-Llama-70BParadigm=LLM-as-a-Judge & Generative Reward Models, Sampling Strategy=Maj@322025.07 | 67 | — | — | |
| DeepSeek-V3.2-thinkingCategory=LLM-as-Judge, Mode=thinking2026.02 | 66.8 | 26.2 | 49.2 | |
| GPT-5-chatCategory=LLM-as-Judge, Mode=chat2026.02 | 66.6 | 19.6 | 53.5 | |
| Skywork-Reward-V2-Llama-3.2-1BParadigm=Our Reward Models2025.07 | 66.6 | — | — | |
| DeepSeek-V3.2-chatCategory=LLM-as-Judge, Mode=chat2026.02 | 66.4 | 24 | 50.5 | |
| J1-Llama-70BParadigm=LLM-as-a-Judge & Generative Reward Models2025.07 | 66.3 | — | — | |
| GenRM-RLVR-14BCategory=Our Methods (Baseline), Parameters=14B, Training=RLVR2026.02 | 65.7 | 45.7 | 35.6 | |
| GenRM-R-Align-14BCategory=Our Methods, Parameters=14B, Training=R-Align2026.02 | 65.7 | 26.9 | 48.1 | |
| RM-R1-Qwen-32BCategory=Specialized Generative Reward Models, Parameters=32B2026.02 | 65.6 | 30 | 45.9 | |
| DeepSeek-GRM-27BParadigm=LLM-as-a-Judge & Generative Reward Models2025.07 | 65.3 | — | — | |
| Skywork-Reward-V2-Qwen3-0.6BParadigm=Our Reward Models2025.07 | 65.3 | — | — | |
| Qwen3-14BCategory=Our Methods (Baseline), Parameters=14B2026.02 | 65.2 | 36.9 | 41.1 | |
| RRM-32BCategory=Specialized Generative Reward Models, Parameters=32B2026.02 | 65.1 | 62.4 | 24.5 | |
| RM-R1-DS-32BCategory=Specialized Generative Reward Models, Parameters=32B2026.02 | 64.9 | 46.5 | 34.7 | |
| GPT-5-thinkingCategory=LLM-as-Judge, Mode=thinking2026.02 | 64.7 | 6.9 | 58.8 | |
| GPT-OSS-120BCategory=LLM-as-Judge, Parameters=120B2026.02 | 64.5 | 40.7 | 38.2 | |
| Llama-3.1-Nemotron-70BParadigm=Bradley-Terry2025.07 | 64.2 | — | — | |
| INF-ORM-Llama3.1-70BParadigm=Bradley-Terry2025.07 | 64.2 | — | — | |
| GenRM-RLVR-8BCategory=Our Methods (Baseline), Parameters=8B, Training=RLVR2026.02 | 63.7 | 47.4 | 33.5 | |
| GenRM-R-Align-8BCategory=Our Methods, Parameters=8B, Training=R-Align2026.02 | 63.6 | 34.9 | 41.4 | |
| Skywork-Reward-Gemma-2-27B-v0.2Paradigm=Bradley-Terry2025.07 | 63.6 | — | — | |
| GRM-llama3-8B-distillParadigm=Bradley-Terry2025.07 | 63.2 | — | — | |
| FsfairX-LLaMA3-RM-v0.1Paradigm=Bradley-Terry2025.07 | 63.1 | — | — | |
| Starling-RM-34BParadigm=Bradley-Terry2025.07 | 62.8 | — | — | |
| Gemini-3-ProCategory=LLM-as-Judge, Tier=Pro2026.02 | 62.7 | 1.6 | 61.6 | |
| LDL-Reward-Gemma-2-27B-v0.1Paradigm=Bradley-Terry2025.07 | 62.4 | — | — | |
| Mean RewardPolicy Model=Qwen2.5-3B-Instruct2026.05 | 62.4 | — | — | |
| Qwen3-8BCategory=Our Methods (Baseline), Parameters=8B2026.02 | 62.2 | 45.2 | 34.1 | |
| Skywork-Reward-Llama-3.1-8B-v0.2Paradigm=Bradley-Terry2025.07 | 62.2 | — | — | |
| Internlm2-7b-rewardParadigm=Bradley-Terry2025.07 | 62.1 | — | — | |
| GRM-Llama3-8B-rewardmodel-ftParadigm=Bradley-Terry2025.07 | 62.1 | — | — | |
| Skywork-Reward-Llama-3.1-8BParadigm=Bradley-Terry2025.07 | 62.1 | — | — | |
| RM-Mistral-7BParadigm=Bradley-Terry2025.07 | 61.8 | — | — | |
| BTRM Qwen2 7b 0613Paradigm=Bradley-Terry2025.07 | 61.8 | — | — | |
| Qwen3-4B-Thinking-2507Category=LLM-as-Judge, Parameters=4B, Mode=Thinking2026.02 | 61.5 | 33.7 | 40.8 | |
| Internlm2-20b-rewardParadigm=Bradley-Terry2025.07 | 61 | — | — | |
| ArmoRM-Llama3-8B-v0.1Paradigm=Bradley-Terry2025.07 | 60.6 | — | — | |
| QRM-Llama3.1-8BParadigm=Bradley-Terry2025.07 | 60.6 | — | — | |
| J1-Llama-8BParadigm=LLM-as-a-Judge & Generative Reward Models, Sampling Strategy=Maj@322025.07 | 60.6 | — | — | |
| Qwen3-4B-Instruct-2507Category=LLM-as-Judge, Parameters=4B, Mode=Instruct2026.02 | 60.4 | 44.8 | 33.3 | |
| Skywork-Reward-Gemma-2-27BParadigm=Bradley-Terry2025.07 | 60.3 | — | — | |
| J1-Llama-8BParadigm=LLM-as-a-Judge & Generative Reward Models2025.07 | 60.3 | — | — | |
| URM-LLaMa-3.1-8BParadigm=Bradley-Terry2025.07 | 60.2 | — | — | |
| Mean RewardPolicy Model=Qwen3-4B-Instruct-25072026.05 | 60 | — | — | |
| GRM-gemma2-2B-rewardmodel-ftParadigm=Bradley-Terry2025.07 | 59.7 | — | — | |
| Eurus-RM-7bParadigm=Bradley-Terry2025.07 | 59.6 | — | — | |
| Llama-3-OffsetBias-RM-8BParadigm=Bradley-Terry2025.07 | 59.2 | — | — | |
| internlm2-1.8b-rewardParadigm=Bradley-Terry2025.07 | 57.3 | — | — | |
| QRM-Llama3.1-8B-v2Paradigm=Bradley-Terry2025.07 | 57.2 | — | — | |
| QRM-Gemma-2-27BParadigm=Bradley-Terry2025.07 | 52.3 | — | — |