# S$^2$-MLPv2: Improved Spatial-Shift MLP Architecture for Vision

Source: [S$^2$-MLPv2: Improved Spatial-Shift MLP Architecture for Vision](<https://www.sota2.com/research/paper/68fa6a7b1e209131cea287ad>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:26:28.956Z

Authors: Tan Yu, Xu Li, Yunfeng Cai, Mingming Sun, Ping Li

Published: 2021-08-02T17:59:02.000Z

## Abstract

Recently, MLP-based vision backbones emerge. MLP-based vision architectures with less inductive bias achieve competitive performance in image recognition compared with CNNs and vision Transformers. Among them, spatial-shift MLP (S$^2$-MLP), adopting the straightforward spatial-shift operation, achieves better performance than the pioneering works including MLP-mixer and ResMLP. More recently, using smaller patches with a pyramid structure, Vision Permutator (ViP) and Global Filter Network (GFNet) achieve better performance than S$^2$-MLP.   In this paper, we improve the S$^2$-MLP vision backbone. We expand the feature map along the channel dimension and split the expanded feature map into several parts. We conduct different spatial-shift operations on split parts.   Meanwhile, we exploit the split-attention operation to fuse these split parts. Moreover, like the counterparts, we adopt smaller-scale patches and use a pyramid structure for boosting the image recognition accuracy. We term the improved spatial-shift MLP vision backbone as S$^2$-MLPv2. Using 55M parameters, our medium-scale model, S$^2$-MLPv2-Medium achieves an $83.6\\%$ top-1 accuracy on the ImageNet-1K benchmark using $224\\times 224$ images without self-attention and external training data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2108.01072>)

[PDF](<https://arxiv.org/pdf/2108.01072.pdf>)
