# Bayesian Hypernetworks

Source: [Bayesian Hypernetworks](<https://www.sota2.com/research/paper/68fa64431e209131cea03950>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:00:11.434Z

Authors: David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, Aaron Courville

Published: 2017-10-13T00:27:57.000Z

## Abstract

We study Bayesian hypernetworks: a framework for approximate Bayesian inference in neural networks. A Bayesian hypernetwork $\\h$ is a neural network which learns to transform a simple noise distribution, $p(\\vec\\epsilon) = \\N(\\vec 0,\\mat I)$, to a distribution $q(\\pp) := q(h(\\vec\\epsilon))$ over the parameters $\\pp$ of another neural network (the "primary network")\\@. We train $q$ with variational inference, using an invertible $\\h$ to enable efficient estimation of the variational lower bound on the posterior $p(\\pp \| \\D)$ via sampling. In contrast to most methods for Bayesian deep learning, Bayesian hypernets can represent a complex multimodal approximate posterior with correlations between parameters, while enabling cheap iid sampling of~$q(\\pp)$. In practice, Bayesian hypernets can provide a better defense against adversarial examples than dropout, and also exhibit competitive performance on a suite of tasks which evaluate model uncertainty, including regularization, active learning, and anomaly detection.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [OOD Detection on Dirty-MNIST vs Fashion MNIST (test)](<https://www.sota2.com/research/sota/ood-detection-on-dirty-mnist-vs-fashion-mnist-test.md>): OOD Detection; Dirty-MNIST vs Fashion MNIST (test); NLL: 0.278.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1710.04759>)

[PDF](<https://arxiv.org/pdf/1710.04759.pdf>)
