Reward Modeling on RewardBench (Acc, Brier, CE)
88.8AccuracyTwo-anchor-DeepSeek-Pro
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 88.8 | 0.1169 | 0.3859 | |
| GaussianBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 88.2 | 0.1192 | 0.3897 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 88 | 0.1287 | 0.4082 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 87.6 | 0.1317 | 0.4176 | |
| BTBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 87.1 | 0.1297 | 0.4155 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 85.5 | 0.132 | 0.413 | |
| BT(Hard)Backbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 85.1 | 0.141 | 0.4381 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 84.8 | 0.14 | 0.444 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 83.8 | 0.141 | 0.457 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 83.1 | 0.1582 | 0.4831 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 83 | 0.151 | 0.467 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 82.9 | 0.1507 | 0.4589 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 82.7 | 0.147 | 0.506 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 82.1 | 0.163 | 0.49 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 82.1 | 0.157 | 0.483 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 81.9 | 0.161 | 0.52 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 81.1 | 0.149 | 0.509 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 80.7 | 0.1643 | 0.4958 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 80.4 | 0.167 | 0.502 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 80.3 | 0.181 | 0.546 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 79.9 | 0.167 | 0.551 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 79.4 | 0.169 | 0.59 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 78.7 | 0.168 | 0.644 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 78.4 | 0.18 | 0.534 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 78.4 | 0.188 | 0.558 | |
| GaussianBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 78.3 | 0.1784 | 0.5247 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 77.7 | 0.178 | 0.639 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 77.1 | 0.192 | 0.566 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 76.8 | 0.209 | 0.625 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 75.9 | 0.194 | 0.703 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 74.2 | 0.245 | 1.114 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 73.5 | 0.22 | 0.644 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 73.2 | 0.201 | 0.584 | |
| BTBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 72.6 | 0.2027 | 0.584 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 71.7 | 0.209 | 0.604 | |
| BT(Hard)Backbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 69 | 0.2438 | 0.8986 |