LLM Alignment on OpenAssistant (test)
11.85GPT-4o Win RateDPPrefSyn
Evaluation results
| Method | Links | |
|---|---|---|
| DPPrefSynPrivacy budget (ε)=4, Pref. Alignment=SFT+DPO2026.05 | 11.85 | |
| DPPrefSynPrivacy budget (ε)=4, Pref. Alignment=SFT+PPO2026.05 | 10.11 | |
| DPPrefSynPrivacy budget (ε)=4, Pref. Alignment=SFT2026.05 | 9.33 | |
| DP-FTPrivacy budget (ε)=∞, Pref. Alignment=SFT+DPO2026.05 | 8.2 | |
| DP-FTPrivacy budget (ε)=∞, Pref. Alignment=SFT+PPO2026.05 | 7.08 | |
| DP-FTPrivacy budget (ε)=∞, Pref. Alignment=SFT2026.05 | 4.75 | |
| Base LLMPrivacy budget (ε)=02026.05 | 2.11 |