Reward Modeling on MT-Bench OOD (test)
73ScoreGRM w/ sft
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRM w/ sftBase Model=gemma-2B-it, Training Data Size=40K2024.06 | 73 | — | |
| GRM w/ dpo-norefBase Model=gemma-2B-it, Training Data Size=40K2024.06 | 72.1 | — | |
| Classifier + label smoothBase Model=gemma-2B-it, Training Data Size=40K2024.06 | 71.9 | — | |
| GRM w/ dpoBase Model=gemma-2B-it, Training Data Size=40K2024.06 | 71.3 | — | |
| Classifier + EnsembleBase Model=gemma-2B-it, Training Data Size=40K2024.06 | 71.1 | — | |
| Classifier + marginBase Model=gemma-2B-it, Training Data Size=40K2024.06 | 71 | — | |
| Classifier (baseline)Base Model=gemma-2B-it, Training Data Size=40K2024.06 | 69.1 | — | |
| Classifier (Frozen)Base Model=gemma-2B-it, Training Data Size=40K2024.06 | 68.2 | — | |
| Classifier (baseline)Training Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 71.2 | |
| Classifier (Frozen)Training Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 69.5 | |
| Classifier + EnsembleTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 73.7 | |
| Classifier + label smoothTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 71.2 | |
| Classifier + marginTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 72.6 | |
| GRM w/ dpoTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 73.4 | |
| GRM w/ dpo-norefTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 73 | |
| GRM w/ sftTraining Data Size=400K, Base Model=gemma-2B-it2024.06 | — | 73.4 |