ResearchBenchmarksOffline Constrained RLHF on PKU-SafeRLHF 74k (train)Offline Constrained RLHF on PKU-SafeRLHF 74k (train)Follow-1.6034Safety Expected Reward (r1)πλ-1.911552-1.831551-1.75155-1.671549Mar 31, 2026Evaluation resultsMethodMethodLinksSafety Expected Reward (r1)Helpfulness Expected Reward (r2)Violation RateπλPolicy Type=Dual-only...Policy Type=Dual-only learned policy, Jmin=−0.6473, η=0.3, T=10002026.03-1.6034-0.64790.06π0Policy Type=Reference...Policy Type=Reference Policy, Jmin=−0.6473, η=0.3, T=10002026.03-1.8997-0.886723.94