# No Preference Left Behind: Group Distributional Preference Optimization Source: [No Preference Left Behind: Group Distributional Preference Optimization]() Collection index: [llms.txt]() Data updated: 2025-10-26T04:40:15.707Z Authors: Binwei Yao, Zefan Cai, Yun-Shiuan Chuang, Shanglin Yang, Ming Jiang, Diyi Yang, Junjie Hu Published: 2024-12-28T23:30:47.000Z ## Abstract Preferences within a group of people are not uniform but follow a distribution. While existing alignment methods like Direct Preference Optimization (DPO) attempt to steer models to reflect human preferences, they struggle to capture the distributional pluralistic preferences within a group. These methods often skew toward dominant preferences, overlooking the diversity of opinions, especially when conflicting preferences arise. To address this issue, we propose Group Distributional Preference Optimization (GDPO), a novel framework that aligns language models with the distribution of preferences within a group by incorporating the concept of beliefs that shape individual preferences. GDPO calibrates a language model using statistical estimation of the group's belief distribution and aligns the model with belief-conditioned preferences, offering a more inclusive alignment framework than traditional methods. In experiments using both synthetic controllable opinion generation and real-world movie review datasets, we show that DPO fails to align with the targeted belief distributions, while GDPO consistently reduces this alignment gap during training. Moreover, our evaluation metrics demonstrate that GDPO outperforms existing approaches in aligning with group distributional preferences, marking a significant advance in pluralistic alignment. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Language Generation on KGRec-Music (test)](): Language Generation; KGRec-Music (test); Rouge-1: 38.45. - [Language Generation on KGRec (test)](): Language Generation; KGRec (test); ROUGE-1: 38.45. - [Language Generation on Synthetic Data (test)](): Language Generation; Synthetic data (test); ROUGE-1: 56.63. - [Language Generation on Goodreads Books (test)](): Language Generation; Goodreads Books (test); ROUGE-1: 49.87. ## Original sources and implementation [Original paper]() [PDF]()