# Pruning On-the-Fly: A Recoverable Pruning Method without Fine-tuning

Source: [Pruning On-the-Fly: A Recoverable Pruning Method without Fine-tuning](<https://www.sota2.com/research/paper/68fa6e401e209131cea3eb0a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:53:04.613Z

Authors: Dan Liu, Xue Liu

Published: 2022-12-24T04:33:03.000Z

## Abstract

Most existing pruning works are resource-intensive, requiring retraining or fine-tuning of the pruned models for accuracy. We propose a retraining-free pruning method based on hyperspherical learning and loss penalty terms. The proposed loss penalty term pushes some of the model weights far from zero, while the rest weight values are pushed near zero and can be safely pruned with no need for retraining and a negligible accuracy drop. In addition, our proposed method can instantly recover the accuracy of a pruned model by replacing the pruned values with their mean value. Our method obtains state-of-the-art results in retraining-free pruning and is evaluated on ResNet-18/50 and MobileNetV2 with ImageNet dataset. One can easily get a 50\\% pruned ResNet18 model with a 0.47\\% accuracy drop. With fine-tuning, the experiment results show that our method can significantly boost the accuracy of the pruned models compared with existing works. For example, the accuracy of a 70\\% pruned (except the first convolutional layer) MobileNetV2 model only drops 3.5\\%, much less than the 7\\% $\\sim$ 10\\% accuracy drop with conventional methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2212.12651>)

[PDF](<https://arxiv.org/pdf/2212.12651.pdf>)
