Reward Modeling on HHH-Alignment (OOD)
79.8AccuracyGRM w/ sft
Evaluation results
| Method | Links | |
|---|---|---|
| GRM w/ sftTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | 79.8 | |
| GRM w/ dpo-norefTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | 79.7 | |
| GRM w/ dpoTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | 79.2 | |
| Classifier + EnsembleTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | 76.8 | |
| Classifier + marginTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | 75 | |
| Classifier (baseline)Training Data Size=400K, Base Model=gemma-2B-it2024.06 | 73.4 | |
| Classifier + label smoothTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | 72.1 | |
| Classifier (Frozen)Training Data Size=400K, Base Model=gemma-2B-it2024.06 | 66.4 |