# Self-Supervised Disentanglement by Leveraging Structure in Data Augmentations

Source: [Self-Supervised Disentanglement by Leveraging Structure in Data Augmentations](<https://www.sota2.com/research/paper/68fa74821e209131cea620be>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:11:37.234Z

Authors: Cian Eastwood, Julius von K\\"ugelgen, Linus Ericsson, Diane Bouchacourt, Pascal Vincent, Bernhard Sch\\"olkopf, Mark Ibrahim

Published: 2023-11-15T09:34:08.000Z

## Abstract

Self-supervised representation learning often uses data augmentations to induce some invariance to "style" attributes of the data. However, with downstream tasks generally unknown at training time, it is difficult to deduce a priori which attributes of the data are indeed "style" and can be safely discarded. To deal with this, current approaches try to retain some style information by tuning the degree of invariance to some particular task, such as ImageNet object classification. However, prior work has shown that such task-specific tuning can lead to significant performance degradation on other tasks that rely on the discarded style. To address this, we introduce a more principled approach that seeks to disentangle style features rather than discard them. The key idea is to add multiple style embedding spaces where: (i) each is invariant to all-but-one augmentation; and (ii) joint entropy is maximized. We formalize our structured data-augmentation procedure from a causal latent-variable-model perspective, and prove identifiability of both content and individual style variables. We empirically demonstrate the benefits of our approach on both synthetic and real-world data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2311.08815>)

[PDF](<https://arxiv.org/pdf/2311.08815.pdf>)
