# $S^{2}$-LBI: Stochastic Split Linearized Bregman Iterations for Parsimonious Deep Learning

Source: [$S^{2}$-LBI: Stochastic Split Linearized Bregman Iterations for Parsimonious Deep Learning](<https://www.sota2.com/research/paper/68fa65aa1e209131cea0bc65>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:32:37.817Z

Authors: Yanwei Fu, Donghao Li, Xinwei Sun, Shun Zhang, Yizhou Wang, Yuan Yao

Published: 2019-04-24T15:31:55.000Z

## Abstract

This paper proposes a novel Stochastic Split Linearized Bregman Iteration ($S^{2}$-LBI) algorithm to efficiently train the deep network. The $S^{2}$-LBI introduces an iterative regularization path with structural sparsity. Our $S^{2}$-LBI combines the computational efficiency of the LBI, and model selection consistency in learning the structural sparsity. The computed solution path intrinsically enables us to enlarge or simplify a network, which theoretically, is benefited from the dynamics property of our $S^{2}$-LBI algorithm. The experimental results validate our $S^{2}$-LBI on MNIST and CIFAR-10 dataset. For example, in MNIST, we can either boost a network with only 1.5K parameters (1 convolutional layer of 5 filters, and 1 FC layer), achieves 98.40\\% recognition accuracy; or we simplify $82.5\\%$ of parameters in LeNet-5 network, and still achieves the 98.47\\% recognition accuracy. In addition, we also have the learning results on ImageNet, which will be added in the next version of our report.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1904.10873>)

[PDF](<https://arxiv.org/pdf/1904.10873.pdf>)