Reward Model Bias Evaluation on Biased preference dataset
9.75Mean Reward (Total)URM
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| URMmodel_type=external model2026.05 | 9.75 | 2.43 | 8.96 | 10.53 | 12.9 | |
| RMtraining_source=biased preference dataset2026.05 | 5.54 | 1.68 | 5.84 | 5.23 | 76.9 | |
| Skyworkmodel_type=external model2026.05 | 5.44 | 9.39 | 3.18 | 7.7 | 24.2 | |
| InfoRMtraining_source=biased preference dataset2026.05 | 3.7 | 1.2 | 4 | 3.4 | 87 | |
| WARMtraining_source=biased preference dataset2026.05 | 3.12 | 0.97 | 3.44 | 2.8 | 96.8 | |
| QRMmodel_type=external model2026.05 | 2.93 | 0.68 | 2.75 | 3.11 | 30.3 | |
| RRMtraining_source=biased preference dataset2026.05 | 0.27 | 1.75 | 0.41 | 0.14 | 62.6 | |
| SARMmodel_type=external model2026.05 | -0.06 | 0.04 | -0.07 | -0.04 | 13.8 | |
| DPOtraining_source=biased preference dataset2026.05 | -2.42 | 1.21 | -2.16 | -2.68 | 74.4 |