# Winner-Take-All Autoencoders

Source: [Winner-Take-All Autoencoders](<https://www.sota2.com/research/paper/68fa61e61e209131ce9f6839>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:08:28.563Z

Authors: Alireza Makhzani, Brendan Frey

Published: 2014-09-09T14:38:43.000Z

## Abstract

In this paper, we propose a winner-take-all method for learning hierarchical sparse representations in an unsupervised fashion. We first introduce fully-connected winner-take-all autoencoders which use mini-batch statistics to directly enforce a lifetime sparsity in the activations of the hidden units. We then propose the convolutional winner-take-all autoencoder which combines the benefits of convolutional architectures and autoencoders for learning shift-invariant sparse representations. We describe a way to train convolutional autoencoders layer by layer, where in addition to lifetime sparsity, a spatial sparsity within each feature map is achieved using winner-take-all activation functions. We will show that winner-take-all autoencoders can be used to to learn deep sparse representations from the MNIST, CIFAR-10, ImageNet, Street View House Numbers and Toronto Face datasets, and achieve competitive classification performance.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Image Classification on MNIST standard (test)](<https://www.sota2.com/research/sota/image-classification-on-mnist-standard-test.md>): Image Classification; MNIST standard (test); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1409.2752>)

[PDF](<https://arxiv.org/pdf/1409.2752.pdf>)