# Parallel Sequence Modeling via Generalized Spatial Propagation Network

Source: [Parallel Sequence Modeling via Generalized Spatial Propagation Network](<https://www.sota2.com/research/paper/68fa76a61e209131cea6e2c6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:58:19.729Z

Authors: Hongjun Wang, Wonmin Byeon, Jiarui Xu, Jinwei Gu, Ka Chun Cheung, Xiaolong Wang, Kai Han, Jan Kautz, Sifei Liu

Published: 2025-01-21T18:56:19.000Z

## Abstract

We present the Generalized Spatial Propagation Network (GSPN), a new attention mechanism optimized for vision tasks that inherently captures 2D spatial structures. Existing attention models, including transformers, linear attention, and state-space models like Mamba, process multi-dimensional data as 1D sequences, compromising spatial coherence and efficiency. GSPN overcomes these limitations by directly operating on spatially coherent image data and forming dense pairwise connections through a line-scan approach. Central to GSPN is the Stability-Context Condition, which ensures stable, context-aware propagation across 2D sequences and reduces the effective sequence length to $\\sqrt{N}$ for a square map with N elements, significantly enhancing computational efficiency. With learnable, input-dependent weights and no reliance on positional embeddings, GSPN achieves superior spatial fidelity and state-of-the-art performance in vision tasks, including ImageNet classification, class-guided image generation, and text-to-image generation. Notably, GSPN accelerates SD-XL with softmax-attention by over $84\\times$ when generating 16K images.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet-1K (val) (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-val-accuracy.md>): Image Classification; ImageNet-1k (val); Top-1 Accuracy: 84.3.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2501.12381>)

[PDF](<https://arxiv.org/pdf/2501.12381.pdf>)