Sylvester Normalizing Flows for Variational Inference

About

Variational inference relies on flexible approximate posterior distributions. Normalizing flows provide a general recipe to construct flexible variational posteriors. We introduce Sylvester normalizing flows, which can be seen as a generalization of planar flows. Sylvester normalizing flows remove the well-known single-unit bottleneck from planar flows, making a single transformation much more flexible. We compare the performance of Sylvester normalizing flows against planar flows and inverse autoregressive flows and demonstrate that they compare favorably on several datasets.

Rianne van den Berg, Leonard Hasenclever, Jakub M. Tomczak, Max Welling · 2018

Related benchmarks

Task	Dataset	Result
Variational Inference	MNIST (test)	Negative ELBO83.32	52
Variational Inference	Omniglot (test)	Test Log Likelihood—	30
Variational Inference	MNIST statically binarized	Negative ELBO83.32	5
Variational Inference	Omniglot	Neg ELBO99	5
Variational Inference	Caltech 101 Silhouettes	Negative ELBO104.62	5
Variational Inference	Caltech Silhouettes (test)	Negative ELBO (nats)104.62	5
Variational Inference	Frey Faces (test)	Negative ELBO (bits/dim)4.45	5
Variational Inference	Freyfaces	Negative ELBO4.45	5