# Trained Ternary Quantization

Source: [Trained Ternary Quantization](<https://www.sota2.com/research/paper/68fa631d1e209131ce9fce8b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:33:20.017Z

Authors: Chenzhuo Zhu, Song Han, Huizi Mao, William J. Dally

Published: 2016-12-04T05:00:22.000Z

## Abstract

Deep neural networks are widely used in machine learning applications. However, the deployment of large neural networks models can be difficult to deploy on mobile devices with limited power budgets. To solve this problem, we propose Trained Ternary Quantization (TTQ), a method that can reduce the precision of weights in neural networks to ternary values. This method has very little accuracy degradation and can even improve the accuracy of some models (32, 44, 56-layer ResNet) on CIFAR-10 and AlexNet on ImageNet. And our AlexNet model is trained from scratch, which means it's as easy as to train normal full precision model. We highlight our trained quantization method that can learn both ternary values and ternary assignment. During inference, only ternary values (2-bit weights) and scaling factors are needed, therefore our models are nearly 16x smaller than full-precision models. Our ternary models can also be viewed as sparse binary weight networks, which can potentially be accelerated with custom circuit. Experiments on CIFAR-10 show that the ternary models obtained by trained quantization method outperform full-precision models of ResNet-32,44,56 by 0.04%, 0.16%, 0.36%, respectively. On ImageNet, our model outperforms full-precision AlexNet model by 0.3% of Top-1 accuracy and outperforms previous ternary models by 3%.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on ImageNet ILSVRC-2012 (val)](<https://www.sota2.com/research/sota/image-classification-on-imagenet-ilsvrc-2012-val.md>): Image Classification; ImageNet ILSVRC-2012 (val); Top-1 Accuracy: 66.6.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1612.01064>)

[PDF](<https://arxiv.org/pdf/1612.01064.pdf>)