# Implicit Feature Decoupling with Depthwise Quantization Source: [Implicit Feature Decoupling with Depthwise Quantization]() Collection index: [llms.txt]() Data updated: 2025-10-28T13:56:52.603Z Authors: Iordanis Fostiropoulos, Barry Boehm Published: 2022-03-15T17:07:19.000Z ## Abstract Quantization has been applied to multiple domains in Deep Neural Networks (DNNs). We propose Depthwise Quantization (DQ) where $\\textit{quantization}$ is applied to a decomposed sub-tensor along the $\\textit{feature axis}$ of weak statistical dependence. The feature decomposition leads to an exponential increase in $\\textit{representation capacity}$ with a linear increase in memory and parameter cost. In addition, DQ can be directly applied to existing encoder-decoder frameworks without modification of the DNN architecture. We use DQ in the context of Hierarchical Auto-Encoder and train end-to-end on an image feature representation. We provide an analysis on cross-correlation between spatial and channel features and we propose a decomposition of the image feature representation along the channel axis. The improved performance of the depthwise operator is due to the increased representation capacity from implicit feature decoupling. We evaluate DQ on the likelihood estimation task, where it outperforms the previous state-of-the-art on CIFAR-10, ImageNet-32 and ImageNet-64. We progressively train with increasing image size a single hierarchical model that uses 69% less parameters and has a faster convergence than the previous works. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()