# AnyControl: Create Your Artwork with Versatile Control on Text-to-Image Generation

Source: [AnyControl: Create Your Artwork with Versatile Control on Text-to-Image Generation](<https://www.sota2.com/research/paper/68fa74251e209131cea5ffa9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:02:53.994Z

Authors: Yanan Sun, Yanchen Liu, Yinhao Tang, Wenjie Pei, Kai Chen

Published: 2024-06-27T07:40:59.000Z

## Abstract

The field of text-to-image (T2I) generation has made significant progress in recent years, largely driven by advancements in diffusion models. Linguistic control enables effective content creation, but struggles with fine-grained control over image generation. This challenge has been explored, to a great extent, by incorporating additional user-supplied spatial conditions, such as depth maps and edge maps, into pre-trained T2I models through extra encoding. However, multi-control image synthesis still faces several challenges. Specifically, current approaches are limited in handling free combinations of diverse input control signals, overlook the complex relationships among multiple spatial conditions, and often fail to maintain semantic alignment with provided textual prompts. This can lead to suboptimal user experiences. To address these challenges, we propose AnyControl, a multi-control image synthesis framework that supports arbitrary combinations of diverse control signals. AnyControl develops a novel Multi-Control Encoder that extracts a unified multi-modal embedding to guide the generation process. This approach enables a holistic understanding of user inputs, and produces high-quality, faithful results under versatile control signals, as demonstrated by extensive quantitative and qualitative evaluations. Our project page is available in https://any-control.github.io.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Generation on PCB](<https://www.sota2.com/research/sota/image-generation-on-pcb.md>): Image Generation; PCB; FID: 142.01.

- [Gesture Transfer on In-the-wild web-sourced images](<https://www.sota2.com/research/sota/gesture-transfer-on-in-the-wild-web-sourced-images.md>): Gesture Transfer; In-the-wild web-sourced images; PSNR: 10.59.

- [Sketch-to-Image Generation on Sketchy In the Wild](<https://www.sota2.com/research/sota/sketch-to-image-generation-on-sketchy-in-the-wild.md>): Sketch-to-Image Generation; Sketchy In the Wild; FID: 1.41.

- [Sketch-to-image on Sketchy (test)](<https://www.sota2.com/research/sota/sketch-to-image-on-sketchy-test.md>): Sketch-to-image; Sketchy (test); FID: 1.06.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.18958>)

[PDF](<https://arxiv.org/pdf/2406.18958.pdf>)