LLM Alignment on Anthropic-HH (test)
57.53GPT-4o Win RateDPPrefSyn
Evaluation Results
| Method | Links | |
|---|---|---|
| DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.05 | 57.53 | |
| DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.05 | 55.95 | |
| DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.05 | 55.08 | |
| DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.05 | 54.9 | |
| DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.05 | 38.72 | |
| DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.05 | 35 | |
| DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.05 | 31.98 | |
| DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.05 | 29.77 |