# Enhancing the Regularization Effect of Weight Pruning in Artificial Neural Networks

Source: [Enhancing the Regularization Effect of Weight Pruning in Artificial Neural Networks](<https://www.sota2.com/research/paper/68fa644b1e209131cea03c69>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:56:35.708Z

Authors: Brian Bartoldson, Adrian Barbu, Gordon Erlebacher

Published: 2018-05-04T20:39:31.000Z

## Abstract

Artificial neural networks (ANNs) may not be worth their computational/memory costs when used in mobile phones or embedded devices. Parameter-pruning algorithms combat these costs, with some algorithms capable of removing over 90% of an ANN's weights without harming the ANN's performance. Removing weights from an ANN is a form of regularization, but existing pruning algorithms do not significantly improve generalization error. We show that pruning ANNs can improve generalization if pruning targets large weights instead of small weights. Applying our pruning algorithm to an ANN leads to a higher image classification accuracy on CIFAR-10 data than applying the popular regularizer dropout. The pruning couples this higher accuracy with an 85% reduction of the ANN's parameter count.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1805.01930>)

[PDF](<https://arxiv.org/pdf/1805.01930.pdf>)