# Training Sparse Neural Networks

Source: [Training Sparse Neural Networks](<https://www.sota2.com/research/paper/68fa63031e209131ce9fc59d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:31:19.733Z

Authors: Suraj Srinivas, Akshayvarun Subramanya, R. Venkatesh Babu

Published: 2016-11-21T09:24:24.000Z

## Abstract

Deep neural networks with lots of parameters are typically used for large-scale computer vision tasks such as image classification. This is a result of using dense matrix multiplications and convolutions. However, sparse computations are known to be much more efficient. In this work, we train and build neural networks which implicitly use sparse computations. We introduce additional gate variables to perform parameter selection and show that this is equivalent to using a spike-and-slab prior. We experimentally validate our method on both small and large networks and achieve state-of-the-art compression results for sparse neural network models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1611.06694>)

[PDF](<https://arxiv.org/pdf/1611.06694.pdf>)