# Location-Aware Self-Supervised Transformers for Semantic Segmentation

Source: [Location-Aware Self-Supervised Transformers for Semantic Segmentation](<https://www.sota2.com/research/paper/68fa6ec51e209131cea41ea9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:56:57.877Z

Authors: Mathilde Caron, Neil Houlsby, Cordelia Schmid

Published: 2022-12-05T16:24:29.000Z

## Abstract

Pixel-level labels are particularly expensive to acquire. Hence, pretraining is a critical step to improve models on a task like semantic segmentation. However, prominent algorithms for pretraining neural networks use image-level objectives, e.g. image classification, image-text alignment a la CLIP, or self-supervised contrastive learning. These objectives do not model spatial information, which might be sub-optimal when finetuning on downstream tasks with spatial reasoning. In this work, we pretrain network with a location-aware (LOCA) self-supervised method which fosters the emergence of strong dense features. Specifically, we use both a patch-level clustering scheme to mine dense pseudo-labels and a relative location prediction task to encourage learning about object parts and their spatial arrangements. Our experiments show that LOCA pretraining leads to representations that transfer competitively to challenging and diverse semantic segmentation datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Semantic Segmentation on PASCAL VOC 2012 (test)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-2012-test.md>): Semantic segmentation; PASCAL VOC 2012 (test); mIoU: 76.7.

- [Semantic Segmentation on ADE20K (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-ade20k-val.md>): Semantic segmentation; ADE20K (val); mIoU: 47.9.

- [Semantic Segmentation on PASCAL VOC 2012 (val)](<https://www.sota2.com/research/sota/semantic-segmentation-on-pascal-voc-2012-val.md>): Semantic segmentation; PASCAL VOC 2012 (val); Mean IoU: 59.5.

- [Monocular Depth Estimation on NYU v2 (test)](<https://www.sota2.com/research/sota/monocular-depth-estimation-on-nyu-v2-test.md>): Monocular Depth Estimation; NYU v2 (test); Abs Rel: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2212.02400>)

[PDF](<https://arxiv.org/pdf/2212.02400.pdf>)