Preference Modeling on HelpSteer2 held-out (test)
68.4Preference AccuracyMean-Var
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Mean-VarReward Model Type=Distributional, Loss=KL2024.10 | 68.4 | 0.582 | |
| Bradley-TerryReward Model Type=Single-Value, Data Scaling=Agg2024.10 | 68.3 | 0.482 | |
| Bradley-TerryReward Model Type=Single-Value, Data Scaling=All2024.10 | 67.8 | 0.489 | |
| MSE RegressionReward Model Type=Single-Value, Data Scaling=All2024.10 | 67.5 | 0.481 | |
| MSE RegressionReward Model Type=Single-Value, Data Scaling=Agg2024.10 | 66.9 | 0.488 | |
| ClassificationReward Model Type=Distributional, Loss=KL2024.10 | 65.9 | 0.648 | |
| Mean-VarReward Model Type=Distributional, Loss=NLL, Independence=Independent2024.10 | 57.4 | 0.573 |