Auxiliary Deep Generative Models

About

Deep generative models parameterized by neural networks have recently achieved state-of-the-art performance in unsupervised and semi-supervised learning. We extend deep generative models with auxiliary variables which improves the variational approximation. The auxiliary variables leave the generative model unchanged but make the variational distribution more expressive. Inspired by the structure of the auxiliary variable we also propose a model with two stochastic layers and skip connections. Our findings suggest that more expressive and properly specified deep generative models converge faster with better results. We show state-of-the-art performance within semi-supervised learning on MNIST, SVHN and NORB datasets.

Lars Maal\oe, Casper Kaae S\onderby, S\oren Kaae S\onderby, Ole Winther · 2016

Related benchmarks

Task	Dataset	Result
Image Classification	SVHN (test)	Accuracy—	470
Classification	SVHN (test)	Error Rate16.61	182
Image Classification	MNIST standard (test)	Accuracy—	69
Semi-supervised Image Classification	MNIST (test)	Test Error0.0096	31
Semi-supervised classification	MNIST 100 labels	Error Rate0.0096	16
Image Classification	SVHN 1,000 labels (train)	Error Rate (%)16.61	15
Semi-supervised classification	SVHN 1000 labels	Error Rate16.61	14
Image Classification	MNIST permutation-invariant (test)	Incorrect Test Examples Count962	14
Generative Modeling	MNIST permutation-invariant (test)	Log Likelihood-82.97	10
Semi-supervised classification	MNIST 100 labels statically binarized (test)	Error Rate (%)0.96	10