ResearchBenchmarksPreference Alignment on Human ValuesPreference Alignment on Human ValuesFollow72.14Helpfulness (Reward)PLOT65.265667.050368.83570.6197Apr 2, 2026Evaluation resultsMethodMethodLinksHelpfulness (Reward)Humanity (GPT-4)PLOTBackbone=Llama3.2-3B-I...Backbone=Llama3.2-3B-Instruct2026.0472.149.15DPOBackbone=Llama3.2-3B-I...Backbone=Llama3.2-3B-Instruct2026.0470.638.65InstructBackbone=Llama3.2-3B-I...Backbone=Llama3.2-3B-Instruct2026.0465.537.8