# From Layers to States: A State Space Model Perspective to Deep Neural Network Layer Dynamics

Source: [From Layers to States: A State Space Model Perspective to Deep Neural Network Layer Dynamics](<https://www.sota2.com/research/paper/68fa77081e209131cea70a4d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:08:06.137Z

Authors: Qinshuo Liu, Weiqin Zhao, Wei Huang, Yanwen Fang, Lequan Yu, Guodong Li

Published: 2025-02-12T08:12:33.000Z

## Abstract

The depth of neural networks is a critical factor for their capability, with deeper models often demonstrating superior performance. Motivated by this, significant efforts have been made to enhance layer aggregation - reusing information from previous layers to better extract features at the current layer, to improve the representational power of deep neural networks. However, previous works have primarily addressed this problem from a discrete-state perspective which is not suitable as the number of network layers grows. This paper novelly treats the outputs from layers as states of a continuous process and considers leveraging the state space model (SSM) to design the aggregation of layers in very deep neural networks. Moreover, inspired by its advancements in modeling long sequences, the Selective State Space Models (S6) is employed to design a new module called Selective State Space Model Layer Aggregation (S6LA). This module aims to combine traditional CNN or transformer architectures within a sequential framework, enhancing the representational capabilities of state-of-the-art vision networks. Extensive experiments show that S6LA delivers substantial improvements in both image classification and detection tasks, highlighting the potential of integrating SSMs with contemporary deep learning techniques.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Object Detection on COCO 2017](<https://www.sota2.com/research/sota/object-detection-on-coco-2017.md>): Object Detection; COCO 2017; AP (Box): 40.6.

- [Image Classification on CIFAR-100 (Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-accuracy.md>): Image Classification; CIFAR-100; Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2502.10463>)

[PDF](<https://arxiv.org/pdf/2502.10463.pdf>)
