# Training Quantized Neural Networks with a Full-precision Auxiliary Module

Source: [Training Quantized Neural Networks with a Full-precision Auxiliary Module](<https://www.sota2.com/research/paper/68fa67351e209131cea14c75>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T22:08:20.129Z

Authors: Bohan Zhuang, Lingqiao Liu, Mingkui Tan, Chunhua Shen, Ian Reid

Published: 2019-03-27T03:42:31.000Z

## Abstract

In this paper, we seek to tackle a challenge in training low-precision networks: the notorious difficulty in propagating gradient through a low-precision network due to the non-differentiable quantization function. We propose a solution by training the low-precision network with a fullprecision auxiliary module. Specifically, during training, we construct a mix-precision network by augmenting the original low-precision network with the full precision auxiliary module. Then the augmented mix-precision network and the low-precision network are jointly optimized. This strategy creates additional full-precision routes to update the parameters of the low-precision model, thus making the gradient back-propagates more easily. At the inference time, we discard the auxiliary module without introducing any computational complexity to the low-precision network. We evaluate the proposed method on image classification and object detection over various quantization approaches and show consistent performance increase. In particular, we achieve near lossless performance to the full-precision model by using a 4-bit detector, which is of great practical value.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-val.md>): Image Classification; ImageNet (val); Top-1 Acc: 73.8.

- [Image Classification on ImageNet 1k (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-val.md>): Image Classification; ImageNet-1k (val); Top-1 Accuracy: 75.4.

- [Object Detection on COCO 2017 (val)](<https://www.sota2.com/research/sota/object-detection-on-coco-2017-val.md>): Object Detection; COCO 2017 (val); AP: 36.1.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1903.11236>)

[PDF](<https://arxiv.org/pdf/1903.11236.pdf>)
