# Quantization Networks

Source: [Quantization Networks](<https://www.sota2.com/research/paper/68fa66b11e209131cea11dad>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:56:21.324Z

Authors: Jiwei Yang, Xu Shen, Jun Xing, Xinmei Tian, Houqiang Li, Bing Deng, Jianqiang Huang, Xiansheng Hua

Published: 2019-11-21T13:44:03.000Z

## Abstract

Although deep neural networks are highly effective, their high computational and memory costs severely challenge their applications on portable devices. As a consequence, low-bit quantization, which converts a full-precision neural network into a low-bitwidth integer version, has been an active and promising research topic. Existing methods formulate the low-bit quantization of networks as an approximation or optimization problem. Approximation-based methods confront the gradient mismatch problem, while optimization-based methods are only suitable for quantizing weights and could introduce high computational cost in the training stage. In this paper, we propose a novel perspective of interpreting and implementing neural network quantization by formulating low-bit quantization as a differentiable non-linear function (termed quantization function). The proposed quantization function can be learned in a lossless and end-to-end manner and works for any weights and activations of neural networks in a simple and uniform way. Extensive experiments on image classification and object detection tasks show that our quantization networks outperform the state-of-the-art methods. We believe that the proposed method will shed new insights on the interpretation of neural network quantization. Our code is available at https://github.com/aliyun/alibabacloud-quantization-networks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet ILSVRC-2012 (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-ilsvrc-2012-val.md>): Image Classification; ImageNet ILSVRC-2012 (val); Top-1 Accuracy: 76.2.

- [Image Classification on ImageNet (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-val.md>): Image Classification; ImageNet (val); Top-1 Acc: 53.6.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1911.09464>)

[PDF](<https://arxiv.org/pdf/1911.09464.pdf>)
