# Vision Transformer with Sparse Scan Prior

Source: [Vision Transformer with Sparse Scan Prior](<https://www.sota2.com/research/paper/68fa7b471e209131cea87814>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:36:41.717Z

Authors: Yuguang Zhang, Qihang Fan, Huaibo Huang

Published: 2024-05-22T04:34:36.000Z

## Abstract

In recent years, Transformers have achieved remarkable progress in computer vision tasks. However, their global modeling often comes with substantial computational overhead, in stark contrast to the human eye's efficient information processing. Inspired by the human eye's sparse scanning mechanism, we propose a \\textbf{S}parse \\textbf{S}can \\textbf{S}elf-\\textbf{A}ttention mechanism ($\\rm{S}^3\\rm{A}$). This mechanism predefines a series of Anchors of Interest for each token and employs local attention to efficiently model the spatial information around these anchors, avoiding redundant global modeling and excessive focus on local information. This approach mirrors the human eye's functionality and significantly reduces the computational load of vision models. Building on $\\rm{S}^3\\rm{A}$, we introduce the \\textbf{S}parse \\textbf{S}can \\textbf{Vi}sion \\textbf{T}ransformer (SSViT). Extensive experiments demonstrate the outstanding performance of SSViT across a variety of tasks. Specifically, on ImageNet classification, without additional supervision or training data, SSViT achieves top-1 accuracies of \\textbf{84.4\\%/85.7\\%} with \\textbf{4.4G/18.2G} FLOPs. SSViT also excels in downstream tasks such as object detection, instance segmentation, and semantic segmentation. Its robustness is further validated across diverse datasets.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2405.13335>)

[PDF](<https://arxiv.org/pdf/2405.13335.pdf>)
