List your company
ResearchFormerly WizWand

LLM Alignment on OpenAssistant

87.08GPT-4o Win RateDPPrefSyn
0.635223.077645.5267.9624May 29, 2026
Updated 5mo ago

Evaluation results

MethodLinks
DPPrefSynDownstream LLM=Qwen-3-4B, Generator LLM=Qwen-3-4B-Instruct, epsilon=∞2026.0587.08
DPPrefSynDownstream LLM=Qwen-3-4B, Generator LLM=Qwen-3-4B-Instruct, epsilon=42026.0586.52
DPPrefSynDownstream LLM=Llama-3-8B, Generator LLM=Qwen-3-4B-Instruct, epsilon=∞2026.0561.52
DPPrefSynDownstream LLM=Llama-3-8B, Generator LLM=Qwen-3-4B-Instruct, epsilon=42026.0560.25
DP-FTDownstream LLM=Qwen-3-4B, epsilon=∞2026.0559.41
DP-FTDownstream LLM=Llama-3-8B, epsilon=∞2026.0559.13
DP-FTDownstream LLM=Qwen-3-4B, epsilon=42026.0554.92
DPPrefSynDownstream LLM=Qwen-3-4B, Generator LLM=Llama-3-8B-Instruct, epsilon=∞2026.0552.11
DPPrefSynDownstream LLM=Qwen-3-4B, Generator LLM=Llama-3-8B-Instruct, epsilon=42026.0551.69
Qwen-3-4BDownstream LLM=Qwen-3-4B, epsilon=02026.0548.17
DP-FTDownstream LLM=Llama-3-8B, epsilon=42026.0545.79
DPPrefSynDownstream LLM=Llama-3-8B, Generator LLM=Llama-3-8B-Instruct, epsilon=∞2026.0544.1
DPPrefSynDownstream LLM=Llama-3-8B, Generator LLM=Llama-3-8B-Instruct, epsilon=42026.0542.28
DPPrefSynDownstream LLM=Gemma-2-2B, Generator LLM=Qwen-3-4B-Instruct, epsilon=∞2026.0537.08
DPPrefSynDownstream LLM=Gemma-2-2B, Generator LLM=Llama-3-8B-Instruct, epsilon=∞2026.0536.66
DPPrefSynDownstream LLM=Gemma-2-2B, Generator LLM=Qwen-3-4B-Instruct, epsilon=42026.0536.24
DPPrefSynDownstream LLM=Gemma-2-2B, Generator LLM=Llama-3-8B-Instruct, epsilon=42026.0536.01
DP-FTDownstream LLM=Gemma-2-2B, epsilon=∞2026.0533.85
DP-FTDownstream LLM=Gemma-2-2B, epsilon=42026.0530.06
Llama-3-8BDownstream LLM=Llama-3-8B, epsilon=02026.0516.15
DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.0512.36
DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=8, Downstream Model=Pythia-2.8B2026.0512.05
DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=4, Downstream Model=Pythia-2.8B2026.0511.85
DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=2, Downstream Model=Pythia-2.8B2026.0511.04
DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=1, Downstream Model=Pythia-2.8B2026.0510.31
DPPrefSynData Type=Synthetic, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.059.86
DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.059.8
DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=8, Downstream Model=Pythia-2.8B2026.059.63
DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=4, Downstream Model=Pythia-2.8B2026.059.33
DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=2, Downstream Model=Pythia-2.8B2026.059.1
DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=1, Downstream Model=Pythia-2.8B2026.059.07
DPPrefSynData Type=Synthetic, Preference Alignment=SFT, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.058.96
DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.058.2
Gemma-2-2BDownstream LLM=Gemma-2-2B, epsilon=02026.057.3
DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=8, Downstream Model=Pythia-2.8B2026.056.32
DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=2, Downstream Model=Pythia-2.8B2026.056.18
DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=4, Downstream Model=Pythia-2.8B2026.056.12
DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=1, Downstream Model=Pythia-2.8B2026.056.04
DP-FTData Type=Original, Preference Alignment=SFT+DPO, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.055.65
DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=∞, Downstream Model=Pythia-2.8B2026.054.75
DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=8, Downstream Model=Pythia-2.8B2026.054.49
DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=4, Downstream Model=Pythia-2.8B2026.054.46
DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=2, Downstream Model=Pythia-2.8B2026.054.38
DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=1, Downstream Model=Pythia-2.8B2026.054.29
DP-FTData Type=Original, Preference Alignment=SFT, Privacy Budget (epsilon)=0.5, Downstream Model=Pythia-2.8B2026.053.96