List your company

Helpfulness alignment on Anthropic hh-rlhf

3.36Gold RewardSHF-CAS
-2.5472-1.01360.522.0536Dec 10, 2025

Evaluation results

MethodLinks
SHF-CASdelta (δ)=1.5, Oracle=Model2025.123.360.610.64
SHF-CASdelta (δ)=1.5, Oracle=GPT-4o2025.123.310.590.67
SHF-CASdelta (δ)=1, Oracle=GPT-4o2025.122.350.810.56
SHF-CASdelta (δ)=1, Oracle=Model2025.122.330.850.55
SHF-CASdelta (δ)=1.2, Oracle=Model2025.122.310.720.52
SHF-CASdelta (δ)=1.2, Oracle=GPT-4o2025.122.180.730.5
Randomdelta (δ)=1.5, Oracle=GPT-4o2025.121.060.960.49
Randomdelta (δ)=1.5, Oracle=Model2025.121.030.980.46
Randomdelta (δ)=1.2, Oracle=GPT-4o2025.120.561.20.41
Randomdelta (δ)=1.2, Oracle=Model2025.120.511.170.42
Randomdelta (δ)=1, Oracle=GPT-4o2025.120.241.30.38
Randomdelta (δ)=1, Oracle=Model2025.120.221.320.36
RSOOracle=Model2025.12-1.021.620.34
PPO2025.12-2.321.850.27