# DeDPO: Debiased Direct Preference Optimization for Diffusion Models Source: [DeDPO: Debiased Direct Preference Optimization for Diffusion Models]() Collection index: [llms.txt]() Data updated: 2026-02-10T05:00:04.897Z Authors: Khiem Pham, Quang Nguyen, Tung Nguyen, Jingsen Zhu, Michele Santacatterina, Dimitris Metaxas, Ramin Zabih Published: 2026-02-05T21:11:00.000Z ## Abstract Direct Preference Optimization (DPO) has emerged as a predominant alignment method for diffusion models, facilitating off-policy training without explicit reward modeling. However, its reliance on large-scale, high-quality human preference labels presents a severe cost and scalability bottleneck. To overcome this, We propose a semi-supervised framework augmenting limited human data with a large corpus of unlabeled pairs annotated via cost-effective synthetic AI feedback. Our paper introduces Debiased DPO (DeDPO), which uniquely integrates a debiased estimation technique from causal inference into the DPO objective. By explicitly identifying and correcting the systematic bias and noise inherent in synthetic annotators, DeDPO ensures robust learning from imperfect feedback sources, including self-training and Vision-Language Models (VLMs). Experiments demonstrate that DeDPO is robust to the variations in synthetic labeling methods, achieving performance that matches and occasionally exceeds the theoretical upper bound of models trained on fully human-labeled data. This establishes DeDPO as a scalable solution for human-AI alignment using inexpensive synthetic supervision. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Text-to-Image Preference Alignment on FiFA-5K (test)](): Text-to-Image Preference Alignment; FiFA-5K (test); PickScore: 22.83. - [Text-to-Image Preference Evaluation on HPD v2 (test)](): Text-to-Image Preference Evaluation; HPD v2 (test); Aesthetic (Mean): 5.74. ## Original sources and implementation [Original paper]() [PDF]()