# Stylizing ViT: Anatomy-Preserving Instance Style Transfer for Domain Generalization

Source: [Stylizing ViT: Anatomy-Preserving Instance Style Transfer for Domain Generalization](<https://www.sota2.com/research/paper/69784676a2b28e761d87c433>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-28T05:00:08.385Z

Authors: Sebastian Doerrich, Francesco Di Salvo, Jonas Alle, Christian Ledig

Published: 2026-01-24T20:53:02.000Z

## Abstract

Deep learning models in medical image analysis often struggle with generalizability across domains and demographic groups due to data heterogeneity and scarcity. Traditional augmentation improves robustness, but fails under substantial domain shifts. Recent advances in stylistic augmentation enhance domain generalization by varying image styles but fall short in terms of style diversity or by introducing artifacts into the generated images. To address these limitations, we propose Stylizing ViT, a novel Vision Transformer encoder that utilizes weight-shared attention blocks for both self- and cross-attention. This design allows the same attention block to maintain anatomical consistency through self-attention while performing style transfer via cross-attention. We assess the effectiveness of our method for domain generalization by employing it for data augmentation on three distinct image classification tasks in the context of histopathology and dermatology. Results demonstrate an improved robustness (up to +13% accuracy) over the state of the art while generating perceptually convincing images without artifacts. Additionally, we show that Stylizing ViT is effective beyond training, achieving a 17% performance improvement during inference when used for test-time augmentation. The source code is available at https://github.com/sdoerrich97/stylizing-vit .

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Disease Classification on Camelyon17-WILDS 1.0 (test)](<https://www.sota2.com/research/sota/disease-classification-on-camelyon17-wilds-1-0-test.md>): Disease Classification; Camelyon17-WILDS 1.0 (test); Test Accuracy: 0.9565.

- [Disease Classification on Epithelium-Stroma 1.0 (test)](<https://www.sota2.com/research/sota/disease-classification-on-epithelium-stroma-1-0-test.md>): Disease Classification; Epithelium-Stroma 1.0 (test); Test Accuracy: 89.07.

- [Style Transfer on Camelyon WILDS 17 (train)](<https://www.sota2.com/research/sota/style-transfer-on-camelyon-wilds-17-train.md>): Style Transfer; Camelyon WILDS 17 (train); FID: 6.2.

- [Style Transfer on Epithelium-Stroma (train)](<https://www.sota2.com/research/sota/style-transfer-on-epithelium-stroma-train.md>): Style Transfer; Epithelium-Stroma (train); FID: 1.5.

- [Style Transfer on Fitzpatrick17k (train)](<https://www.sota2.com/research/sota/style-transfer-on-fitzpatrick17k-train.md>): Style Transfer; Fitzpatrick17k (train); FID: 36.9.

- [Reconstruction on Camelyon17-WILDS (train)](<https://www.sota2.com/research/sota/reconstruction-on-camelyon17-wilds-train.md>): Reconstruction; Camelyon17-WILDS (train); PSNR: 45.4.

- [Reconstruction on Epithelium-Stroma (train)](<https://www.sota2.com/research/sota/reconstruction-on-epithelium-stroma-train.md>): Reconstruction; Epithelium-Stroma (train); PSNR: 39.

- [Reconstruction on Fitzpatrick17k (train)](<https://www.sota2.com/research/sota/reconstruction-on-fitzpatrick17k-train.md>): Reconstruction; Fitzpatrick17k (train); PSNR: 31.5.

- [Disease Classification on Fitzpatrick17k 1.0 (test)](<https://www.sota2.com/research/sota/disease-classification-on-fitzpatrick17k-1-0-test.md>): Disease Classification; Fitzpatrick17k 1.0 (test); Test Accuracy: 80.92.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.17586>)

[PDF](<https://arxiv.org/pdf/2601.17586.pdf>)
