# Layer-Wise Data-Free CNN Compression

Source: [Layer-Wise Data-Free CNN Compression](<https://www.sota2.com/research/paper/68fa6c9e1e209131cea34fc3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:14:46.163Z

Authors: Maxwell Horton, Yanzi Jin, Ali Farhadi, Mohammad Rastegari

Published: 2020-11-18T03:00:05.000Z

## Abstract

We present a computationally efficient method for compressing a trained neural network without using real data. We break the problem of data-free network compression into independent layer-wise compressions. We show how to efficiently generate layer-wise training data using only a pretrained network. We use this data to perform independent layer-wise compressions on the pretrained network. We also show how to precondition the network to improve the accuracy of our layer-wise compression method. We present results for layer-wise compression using quantization and pruning. When quantizing, we compress with higher accuracy than related works while using orders of magnitude less compute. When compressing MobileNetV2 and evaluating on ImageNet, our method outperforms existing methods for quantization at all bit-widths, achieving a $+0.34\\%$ improvement in $8$-bit quantization, and a stronger improvement at lower bit-widths (up to a $+28.50\\%$ improvement at $5$ bits). When pruning, we outperform baselines of a similar compute envelope, achieving $1.5$ times the sparsity rate at the same accuracy. We also show how to combine our efficient method with high-compute generative methods to improve upon their results.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2011.09058>)

[PDF](<https://arxiv.org/pdf/2011.09058.pdf>)
