# Learnable Companding Quantization for Accurate Low-bit Neural Networks

Source: [Learnable Companding Quantization for Accurate Low-bit Neural Networks](<https://www.sota2.com/research/paper/68fa6b271e209131cea2c565>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:05:21.703Z

Authors: Kohei Yamamoto

Published: 2021-03-12T09:06:52.000Z

## Abstract

Quantizing deep neural networks is an effective method for reducing memory consumption and improving inference speed, and is thus useful for implementation in resource-constrained devices. However, it is still hard for extremely low-bit models to achieve accuracy comparable with that of full-precision models. To address this issue, we propose learnable companding quantization (LCQ) as a novel non-uniform quantization method for 2-, 3-, and 4-bit models. LCQ jointly optimizes model weights and learnable companding functions that can flexibly and non-uniformly control the quantization levels of weights and activations. We also present a new weight normalization technique that allows more stable training for quantization. Experimental results show that LCQ outperforms conventional state-of-the-art methods and narrows the gap between quantized and full-precision models for image classification and object detection tasks. Notably, the 2-bit ResNet-50 model on ImageNet achieves top-1 accuracy of 75.1% and reduces the gap to 1.7%, allowing LCQ to further exploit the potential of non-uniform quantization.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet (val) (Top-1/Top-5 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-val-top-1-top-5-accuracy.md>): Image Classification; ImageNet (val); Top-1 Accuracy: 70.8.

- [Image Classification on ImageNet (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-val.md>): Image Classification; ImageNet (val); Top-1 Acc: 76.6.

- [Object Detection on COCO (val)](<https://www.sota2.com/research/sota/object-detection-on-coco-val.md>): Object Detection; COCO (val); mAP: 36.1.

- [Image Classification on ImageNet 1k (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-1k-val.md>): Image Classification; ImageNet-1k (val); Top-1 Accuracy: 76.6.

- [Object Detection on COCO 2017 (val)](<https://www.sota2.com/research/sota/object-detection-on-coco-2017-val.md>): Object Detection; COCO 2017 (val); AP: 37.1.

- [Image Classification on CIFAR-10 (test)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-test.md>): Image Classification; CIFAR-10 (test); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2103.07156>)

[PDF](<https://arxiv.org/pdf/2103.07156.pdf>)
