# Towards Practical Plug-and-Play Diffusion Models

Source: [Towards Practical Plug-and-Play Diffusion Models](<https://www.sota2.com/research/paper/68fa6ee81e209131cea42cf4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:25:09.637Z

Authors: Hyojun Go, Yunsung Lee, Jin-Young Kim, Seunghyun Lee, Myeongho Jeong, Hyun Seung Lee, Seungtaek Choi

Published: 2022-12-12T15:29:46.000Z

## Abstract

Diffusion-based generative models have achieved remarkable success in image generation. Their guidance formulation allows an external model to plug-and-play control the generation process for various tasks without finetuning the diffusion model. However, the direct use of publicly available off-the-shelf models for guidance fails due to their poor performance on noisy inputs. For that, the existing practice is to fine-tune the guidance models with labeled data corrupted with noises. In this paper, we argue that this practice has limitations in two aspects: (1) performing on inputs with extremely various noises is too hard for a single guidance model; (2) collecting labeled datasets hinders scaling up for various tasks. To tackle the limitations, we propose a novel strategy that leverages multiple experts where each expert is specialized in a particular noise range and guides the reverse process of the diffusion at its corresponding timesteps. However, as it is infeasible to manage multiple networks and utilize labeled data, we present a practical guidance framework termed Practical Plug-And-Play (PPAP), which leverages parameter-efficient fine-tuning and data-free knowledge transfer. We exhaustively conduct ImageNet class conditional generation experiments to show that our method can successfully guide diffusion with small trainable parameters and no labeled data. Finally, we show that image classifiers, depth estimators, and semantic segmentation models can guide publicly available GLIDE through our framework in a plug-and-play manner. Our code is available at https://github.com/riiid/PPAP.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Class-conditional image generation on ImageNet 256x256 (val)](<https://www.sota2.com/research/sota/class-conditional-image-generation-on-imagenet-256x256-val.md>): Class-conditional Image Generation; ImageNet 256x256 (val); Inception Score (IS): 83.26.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2212.05973>)

[PDF](<https://arxiv.org/pdf/2212.05973.pdf>)

[Implementation](<https://github.com/riiid/PPAP>)
