# ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On

Source: [ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On](<https://www.sota2.com/research/paper/6a41fc26206542251d7ccc2a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-30T05:00:05.200Z

Authors: Xiangyu Sai, Meysam Madadi, Sergio Escalera, Yong Xu

Published: 2026-06-26T06:59:50.000Z

## Abstract

Image-based virtual try-on has emerged as a compelling task in e-commerce and augmented reality, yet existing methods struggle to simultaneously preserve fine garment semantics and adapt to diverse person body geometries under large clothing-body deformations. We present ModaFlow, a modality-aware flow-matching based framework for high-fidelity virtual try-on that achieves precise alignment between textual descriptions and garment appearance. Unlike prior methods that treat multimodal conditions uniformly, ModaFlow introduces a modality-aware guidance scheme: visual garment embeddings extracted by a pretrained image prompt adapter provide deterministic, persistent structural guidance, while textual embeddings generated from garment descriptions are controlled via classifier-free guidance (CFG) with adaptive scaling and zero-initialized velocity. To further enhance flow field accuracy, we propose two regularization losses, cosine similarity and perceptual flow discrimination, that jointly improve directional consistency and perceptual realism of the velocity field. Additionally, a mask manipulation strategy stochastically samples among box, transparent, and relaxed masks during training, simulating diverse occlusion scenarios and enabling robust inference under unpaired settings where only a box mask is available. Experiments show that ModaFlow achieves state-of-the-art results in both qualitative and quantitative evaluations, reducing FID by approximately 30% on paired and 20% on unpaired benchmarks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Virtual Try-On on VITON-HD Paired 1024x768 (test)](<https://www.sota2.com/research/sota/virtual-try-on-on-viton-hd-paired-1024x768-test.md>): Virtual Try-On; VITON-HD Paired 1024x768 (test); SSIM: 90.5.

- [Virtual Try-On on VITON-HD Unpaired 1024x768 (test)](<https://www.sota2.com/research/sota/virtual-try-on-on-viton-hd-unpaired-1024x768-test.md>): Virtual Try-On; VITON-HD Unpaired 1024x768 (test); FID: 8.041.

- [Virtual Try-On on DressCode Paired 1024x768 (test)](<https://www.sota2.com/research/sota/virtual-try-on-on-dresscode-paired-1024x768-test.md>): Virtual Try-On; DressCode Paired 1024x768 (test); SSIM: 0.93.

- [Virtual Try-On on DressCode Unpaired 1024x768 (test)](<https://www.sota2.com/research/sota/virtual-try-on-on-dresscode-unpaired-1024x768-test.md>): Virtual Try-On; DressCode Unpaired 1024x768 (test); FID: 6.011.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.27773>)

[PDF](<https://arxiv.org/pdf/2606.27773.pdf>)
