# DP-LDMs: Differentially Private Latent Diffusion Models

Source: [DP-LDMs: Differentially Private Latent Diffusion Models](<https://www.sota2.com/research/paper/68fa77451e209131cea721ab>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:13:53.325Z

Authors: Michael F. Liu, Saiyue Lyu, Margarita Vinaroz, Mijung Park

Published: 2023-05-25T06:18:31.000Z

## Abstract

Diffusion models (DMs) are one of the most widely used generative models for producing high quality images. However, a flurry of recent papers points out that DMs are least private forms of image generators, by extracting a significant number of near-identical replicas of training images from DMs. Existing privacy-enhancing techniques for DMs, unfortunately, do not provide a good privacy-utility tradeoff. In this paper, we aim to improve the current state of DMs with differential privacy (DP) by adopting the $\\textit{Latent}$ Diffusion Models (LDMs). LDMs are equipped with powerful pre-trained autoencoders that map the high-dimensional pixels into lower-dimensional latent representations, in which DMs are trained, yielding a more efficient and fast training of DMs. Rather than fine-tuning the entire LDMs, we fine-tune only the $\\textit{attention}$ modules of LDMs with DP-SGD, reducing the number of trainable parameters by roughly $90\\%$ and achieving a better privacy-accuracy trade-off. Our approach allows us to generate realistic, high-dimensional images (256x256) conditioned on text prompts with DP guarantees, which, to the best of our knowledge, has not been attempted before. Our approach provides a promising direction for training more powerful, yet training-efficient differentially private DMs, producing high-quality DP images. Our code is available at https://anonymous.4open.science/r/DP-LDM-4525.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Natural Language Understanding on GLUE (test) (MNLI, QQP, SST-2, SST-5)](<https://www.sota2.com/research/sota/natural-language-understanding-on-glue-test-mnli-qqp-sst-2-sst-5.md>): Natural Language Understanding; GLUE (test); MNLI Score: 77.9.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2305.15759>)

[PDF](<https://arxiv.org/pdf/2305.15759.pdf>)