List your company

Safety Alignment on PKU-SafeRLHF

3.92Gold RewardPPO
-3.828-1.81650.1952.2065Dec 10, 2025

Evaluation results

MethodLinks
PPO2025.123.921.110.042
RSOOracle=Model2025.122.841.140.037
Randomdelta (δ)=1.5, Oracle=Model2025.122.290.370.18
Randomdelta (δ)=1.4, Oracle=Model2025.122.270.0630.0007
Randomdelta (δ)=1.4, Oracle=GPT-4o2025.122.040.14-0.031
Randomdelta (δ)=1.5, Oracle=GPT-4o2025.121.890.590.031
SHF-CASdelta (δ)=1.5, Oracle=Model2025.121.60.0360.32
SHF-CASdelta (δ)=1.4, Oracle=Model2025.121.590.00030.0019
SHF-CASdelta (δ)=1.4, Oracle=GPT-4o2025.121.310.00410.0007
SHF-CASdelta (δ)=1.5, Oracle=GPT-4o2025.121.210.0670.1
Randomdelta (δ)=1.6, Oracle=Model2025.12-0.641.350.026
Randomdelta (δ)=1.6, Oracle=GPT-4o2025.12-1.461.410.017
SHF-CASdelta (δ)=1.6, Oracle=Model2025.12-2.90.160.34
SHF-CASdelta (δ)=1.6, Oracle=GPT-4o2025.12-3.530.350.11