Reward Modeling on PPE-P (Acc, Brier, CE)
63AccuracyTwo-anchor-Skywork
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 63 | 0.244 | 0.782 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 61.8 | 0.2335 | 0.6757 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 61.5 | 0.2303 | 0.6586 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 61.4 | 0.244 | 0.732 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 61.4 | 0.246 | 0.742 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 61.3 | 0.2284 | 0.6523 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 61.3 | 0.2292 | 0.6508 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 61.2 | 0.24 | 0.696 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 61 | 0.2301 | 0.6538 | |
| GaussianBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 60.9 | 0.2326 | 0.66 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 60.9 | 0.267 | 0.862 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 60.8 | 0.238 | 0.68 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 60.7 | 0.241 | 0.688 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 60.5 | 0.2321 | 0.6581 | |
| BTBackbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 60.4 | 0.2341 | 0.663 | |
| BTBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 60.4 | 0.248 | 0.7315 | |
| GaussianBackbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 60.4 | 0.2472 | 0.754 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 60.3 | 0.285 | 1.015 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer3-20K2026.05 | 60.3 | 0.264 | 0.932 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 60.2 | 0.241 | 0.694 | |
| BT(Hard)Backbone=Llama-3.2-8B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 60 | 0.2351 | 0.6688 | |
| Two-anchor-SkyworkBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 60 | 0.234 | 0.664 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 60 | 0.237 | 0.672 | |
| BT(Hard)Backbone=Llama-3.2-8B-Instruct, Training Dataset=MultiPref-8K2026.05 | 59.9 | 0.2748 | 0.911 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 59.5 | 0.305 | 1.251 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 59.5 | 0.253 | 0.725 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 59.4 | 0.234 | 0.66 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 59.3 | 0.236 | 0.665 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=PersonalLLM-40K2026.05 | 59.2 | 0.252 | 0.729 | |
| BTBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 58.8 | 0.251 | 0.719 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 58.4 | 0.284 | 0.932 | |
| Two-anchor-DeepSeek-FlashBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 58.2 | 0.239 | 0.672 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=HelpSteer2-9K2026.05 | 58.2 | 0.251 | 0.729 | |
| GaussianBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 57.7 | 0.242 | 0.68 | |
| BT(Hard)Backbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 57.1 | 0.295 | 0.962 | |
| Two-anchor-DeepSeek-ProBackbone=Llama-3.2-3B-Instruct, Training Dataset=MultiPref-8K2026.05 | 56.3 | 0.244 | 0.682 |