# Learning low-precision neural networks without Straight-Through Estimator(STE) Source: [Learning low-precision neural networks without Straight-Through Estimator(STE)]() Collection index: [llms.txt]() Data updated: 2025-10-25T21:35:37.665Z Authors: Zhi-Gang Liu, Matthew Mattina Published: 2019-03-04T03:47:19.000Z ## Abstract The Straight-Through Estimator (STE) is widely used for back-propagating gradients through the quantization function, but the STE technique lacks a complete theoretical understanding. We propose an alternative methodology called alpha-blending (AB), which quantizes neural networks to low-precision using stochastic gradient descent (SGD). Our method (AB) avoids STE approximation by replacing the quantized weight in the loss function by an affine combination of the quantized weight w\_q and the corresponding full-precision weight w with non-trainable scalar coefficient $\\alpha$ and $1-\\alpha$. During training, $\\alpha$ is gradually increased from 0 to 1; the gradient updates to the weights are through the full-precision term, $(1-\\alpha)w$, of the affine combination; the model is converted from full-precision to low-precision progressively. To evaluate the method, a 1-bit BinaryNet on CIFAR10 dataset and 8-bits, 4-bits MobileNet v1, ResNet\_50 v1/2 on ImageNet dataset are trained using the alpha-blending approach, and the evaluation indicates that AB improves top-1 accuracy by 0.9%, 0.82% and 2.93% respectively compared to the results of STE based quantization. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()