ResearchBenchmarksTwo-objective simultaneous preference alignment on Mix-SafeRLHF-UltraFeedbackTwo-objective simultaneous preference alignment on Mix-SafeRLHF-UltraFeedbackFollow96.54Harmless RateMORA44.737658.186371.63585.0837May 12, 2026Evaluation resultsMethodMethodLinksHarmless RateHelpful Win RateAverage ScoreMORA2026.0596.5479.1386.67MO-ODPO2026.059569.0682.03RSDPO-W2026.0594.0370.0682.04MODPO2026.0569.4275.0372.23MOD2026.0567.368.3267.81DPO-Soups2026.0560.3867.3263.85DPO2026.0556.1571.0563.6SFT2026.0546.735048.73