# Spatial Cross-Attention Improves Self-Supervised Visual Representation Learning

Source: [Spatial Cross-Attention Improves Self-Supervised Visual Representation Learning](<https://www.sota2.com/research/paper/68fa6cc61e209131cea35e4a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:18:38.162Z

Authors: Mehdi Seyfi, Amin Banitalebi-Dehkordi, Yong Zhang

Published: 2022-06-07T21:14:52.000Z

## Abstract

Unsupervised representation learning methods like SwAV are proved to be effective in learning visual semantics of a target dataset. The main idea behind these methods is that different views of a same image represent the same semantics. In this paper, we further introduce an add-on module to facilitate the injection of the knowledge accounting for spatial cross correlations among the samples. This in turn results in distilling intra-class information including feature level locations and cross similarities between same-class instances. The proposed add-on can be added to existing methods such as the SwAV. We can later remove the add-on module for inference without any modification of the learned weights. Through an extensive set of empirical evaluations, we verify that our method yields an improved performance in detecting the class activation maps, top-1 classification accuracy, and down-stream tasks such as object detection, with different configuration settings.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2206.05028>)

[PDF](<https://arxiv.org/pdf/2206.05028.pdf>)
