# Source-Free Domain Adaptation for RGB-D Semantic Segmentation with Vision Transformers

Source: [Source-Free Domain Adaptation for RGB-D Semantic Segmentation with Vision Transformers](<https://www.sota2.com/research/paper/68fa71671e209131cea51362>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:06:39.354Z

Authors: Giulia Rizzoli, Donald Shenaj, Pietro Zanuttigh

Published: 2023-05-23T17:20:47.000Z

## Abstract

With the increasing availability of depth sensors, multimodal frameworks that combine color information with depth data are gaining interest. However, ground truth data for semantic segmentation is burdensome to provide, thus making domain adaptation a significant research area. Yet most domain adaptation methods are not able to effectively handle multimodal data. Specifically, we address the challenging source-free domain adaptation setting where the adaptation is performed without reusing source data. We propose MISFIT: MultImodal Source-Free Information fusion Transformer, a depth-aware framework which injects depth data into a segmentation module based on vision transformers at multiple stages, namely at the input, feature and output levels. Color and depth style transfer helps early-stage domain alignment while re-wiring self-attention between modalities creates mixed features, allowing the extraction of better semantic content. Furthermore, a depth-based entropy minimization strategy is also proposed to adaptively weight regions at different distances. Our framework, which is also the first approach using RGB-D vision transformers for source-free semantic segmentation, shows noticeable performance improvements with respect to standard strategies.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Semantic Segmentation on SYNTHIA to Cityscapes](<https://www.sota2.com/research/sota/semantic-segmentation-on-synthia-to-cityscapes.md>): Semantic segmentation; SYNTHIA to Cityscapes; Road IoU: 80.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2305.14269>)

[PDF](<https://arxiv.org/pdf/2305.14269.pdf>)