# Masked Vector Quantization

Source: [Masked Vector Quantization](<https://www.sota2.com/research/paper/68fa72a11e209131cea57af2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:32:05.519Z

Authors: David D. Nguyen, David Leibowitz, Surya Nepal, Salil S. Kanhere

Published: 2023-01-16T22:30:53.000Z

## Abstract

Generative models with discrete latent representations have recently demonstrated an impressive ability to learn complex high-dimensional data distributions. However, their performance relies on a long sequence of tokens per instance and a large number of codebook entries, resulting in long sampling times and considerable computation to fit the categorical posterior. To address these issues, we propose the Masked Vector Quantization (MVQ) framework which increases the representational capacity of each code vector by learning mask configurations via a stochastic winner-takes-all training regime called Multiple Hypothese Dropout (MH-Dropout). On ImageNet 64$\\times$64, MVQ reduces FID in existing vector quantization architectures by up to $68\\%$ at 2 tokens per instance and $57\\%$ at 5 tokens. These improvements widen as codebook entries is reduced and allows for $7\\textit{--}45\\times$ speed-up in token sampling during inference. As an additional benefit, we find that smaller latent spaces lead to MVQ identifying transferable visual representations where multiple can be smoothly combined.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2301.06626>)

[PDF](<https://arxiv.org/pdf/2301.06626.pdf>)