# DiffuseST: Unleashing the Capability of the Diffusion Model for Style Transfer

Source: [DiffuseST: Unleashing the Capability of the Diffusion Model for Style Transfer](<https://www.sota2.com/research/paper/68fa75541e209131cea673c2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:32:00.805Z

Authors: Ying Hu, Chenyi Zhuang, Pan Gao

Published: 2024-10-19T06:42:43.000Z

## Abstract

Style transfer aims to fuse the artistic representation of a style image with the structural information of a content image. Existing methods train specific networks or utilize pre-trained models to learn content and style features. However, they rely solely on textual or spatial representations that are inadequate to achieve the balance between content and style. In this work, we propose a novel and training-free approach for style transfer, combining textual embedding with spatial features and separating the injection of content or style. Specifically, we adopt the BLIP-2 encoder to extract the textual representation of the style image. We utilize the DDIM inversion technique to extract intermediate embeddings in content and style branches as spatial features. Finally, we harness the step-by-step property of diffusion models by separating the injection of content and style in the target branch, which improves the balance between content preservation and style fusion. Various experiments have demonstrated the effectiveness and robustness of our proposed DiffeseST for achieving balanced and controllable style transfer results, as well as the potential to extend to other tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multi-style Image Transfer on MS-COCO (content) & WikiArt (style) (SD v1.4 backbone, test)](<https://www.sota2.com/research/sota/multi-style-image-transfer-on-ms-coco-content-and-wikiart-style-sd-v1-4-backbone-test.md>): Multi-style Image Transfer; MS-COCO (content) & WikiArt (style) Two-style setting Stable Diffusion v1.4 backbone (test); ArtFID: 20.603.

- [Style Transfer on ArtFID Benchmark (test)](<https://www.sota2.com/research/sota/style-transfer-on-artfid-benchmark-test.md>): Style Transfer; ArtFID Benchmark (test); ArtFID: 18.418.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2410.15007>)

[PDF](<https://arxiv.org/pdf/2410.15007.pdf>)