# Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression

Source: [Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression](<https://www.sota2.com/research/paper/69ddca147d0172140ec799e0>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-06T05:00:11.062Z

Authors: Shiyin Jiang, Wei Long, Minghao Han, Zhenghao Chen, Ce Zhu, Shuhang Gu

Published: 2026-04-12T09:25:24.000Z

## Abstract

The rapid growth of visual data under stringent storage and bandwidth constraints makes extremely low-bitrate image compression increasingly important. While Vector Quantization (VQ) offers strong structural fidelity, existing methods lack a principled mechanism for joint rate-distortion (RD) optimization due to the disconnect between representation learning and entropy modeling. We propose RDVQ, a unified framework that enables end-to-end RD optimization for VQ-based compression via a differentiable relaxation of the codebook distribution, allowing the entropy loss to directly shape the latent prior. We further develop an autoregressive entropy model that supports accurate entropy modeling and test-time rate control. Extensive experiments demonstrate that RDVQ achieves strong performance at extremely low bitrates with a lightweight architecture, attaining competitive or superior perceptual quality with significantly fewer parameters. Compared with RDEIC, RDVQ reduces bitrate by up to 75.71% on DISTS and 37.63% on LPIPS on DIV2K-val. Beyond empirical gains, RDVQ introduces an entropy-constrained formulation of VQ, highlighting the potential for a more unified view of image tokenization and compression. The code will be available at https://github.com/CVL-UESTC/RDVQ.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Compression on Kodak (BD-Rate vs LPIPS/DISTS)](<https://www.sota2.com/research/sota/image-compression-on-kodak-bd-rate-vs-lpips-dists.md>): Image Compression; Kodak; BD-Rate (DISTS): 0.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.10546>)

[PDF](<https://arxiv.org/pdf/2604.10546.pdf>)