# Redistribution of Weights and Activations for AdderNet Quantization

Source: [Redistribution of Weights and Activations for AdderNet Quantization](<https://www.sota2.com/research/paper/68fa6e3b1e209131cea3e7fb>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T05:25:21.303Z

Authors: Ying Nie, Kai Han, Haikang Diao, Chuanjian Liu, Enhua Wu, Yunhe Wang

Published: 2022-12-20T12:24:48.000Z

## Abstract

Adder Neural Network (AdderNet) provides a new way for developing energy-efficient neural networks by replacing the expensive multiplications in convolution with cheaper additions (i.e.l1-norm). To achieve higher hardware efficiency, it is necessary to further study the low-bit quantization of AdderNet. Due to the limitation that the commutative law in multiplication does not hold in l1-norm, the well-established quantization methods on convolutional networks cannot be applied on AdderNets. Thus, the existing AdderNet quantization techniques propose to use only one shared scale to quantize both the weights and activations simultaneously. Admittedly, such an approach can keep the commutative law in the l1-norm quantization process, while the accuracy drop after low-bit quantization cannot be ignored. To this end, we first thoroughly analyze the difference on distributions of weights and activations in AdderNet and then propose a new quantization algorithm by redistributing the weights and the activations. Specifically, the pre-trained full-precision weights in different kernels are clustered into different groups, then the intra-group sharing and inter-group independent scales can be adopted. To further compensate the accuracy drop caused by the distribution difference, we then develop a lossless range clamp scheme for weights and a simple yet effective outliers clamp strategy for activations. Thus, the functionality of full-precision weights and the representation ability of full-precision activations can be fully preserved. The effectiveness of the proposed quantization method for AdderNet is well verified on several benchmarks, e.g., our 4-bit post-training quantized adder ResNet-18 achieves an 66.5% top-1 accuracy on the ImageNet with comparable energy efficiency, which is about 8.5% higher than that of the previous AdderNet quantization methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on CIFAR-100 (W4A4 and W3A3 Performance)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-w4a4-and-w3a3-performance.md>): Image Classification; CIFAR-100; W4A4 Error: -1.83.

- [Image Classification on CIFAR-10 (val)](<https://www.sota2.com/research/sota/image-classification-on-cifar-10-val.md>): Image Classification; CIFAR-10 (val); Top-1 Accuracy: 93.48.

- [Image Classification on ImageNet (val) (Top-1/Top-5 Accuracy)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-val-top-1-top-5-accuracy.md>): Image Classification; ImageNet (val); Top-1 Accuracy: 75.

- [Image Classification on CIFAR-100 (val)](<https://www.sota2.com/research/sota/image-classification-on-cifar-100-val.md>): Image Classification; CIFAR-100 (val); Accuracy: 73.62.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2212.10200>)

[PDF](<https://arxiv.org/pdf/2212.10200.pdf>)

[Implementation](<https://gitee.com/mindspore/models/tree/master/research/cv/AdderQuant>)
