Helpfulness alignment on Anthropic hh-rlhf
3.36Gold RewardSHF-CAS
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SHF-CASdelta (δ)=1.5, Oracle=Model2025.12 | 3.36 | 0.61 | 0.64 | |
| SHF-CASdelta (δ)=1.5, Oracle=GPT-4o2025.12 | 3.31 | 0.59 | 0.67 | |
| SHF-CASdelta (δ)=1, Oracle=GPT-4o2025.12 | 2.35 | 0.81 | 0.56 | |
| SHF-CASdelta (δ)=1, Oracle=Model2025.12 | 2.33 | 0.85 | 0.55 | |
| SHF-CASdelta (δ)=1.2, Oracle=Model2025.12 | 2.31 | 0.72 | 0.52 | |
| SHF-CASdelta (δ)=1.2, Oracle=GPT-4o2025.12 | 2.18 | 0.73 | 0.5 | |
| Randomdelta (δ)=1.5, Oracle=GPT-4o2025.12 | 1.06 | 0.96 | 0.49 | |
| Randomdelta (δ)=1.5, Oracle=Model2025.12 | 1.03 | 0.98 | 0.46 | |
| Randomdelta (δ)=1.2, Oracle=GPT-4o2025.12 | 0.56 | 1.2 | 0.41 | |
| Randomdelta (δ)=1.2, Oracle=Model2025.12 | 0.51 | 1.17 | 0.42 | |
| Randomdelta (δ)=1, Oracle=GPT-4o2025.12 | 0.24 | 1.3 | 0.38 | |
| Randomdelta (δ)=1, Oracle=Model2025.12 | 0.22 | 1.32 | 0.36 | |
| RSOOracle=Model2025.12 | -1.02 | 1.62 | 0.34 | |
| PPO2025.12 | -2.32 | 1.85 | 0.27 |