ResearchDatasetsMix-SafeRLHF-UltraFeedbackMix-SafeRLHF-UltraFeedbackFollowBenchmarksTaskDatasetSOTA methodSortMost resultsRecently updatedMost papersApplyTaskDatasetSOTA resultTrendResultsTwo-objective simultaneous preference alignmentMix-SafeRLHF-UltraFeedback96.54Harmless Rate8