# Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network Source: [Ensemble Model Patching: A Parameter-Efficient Variational Bayesian Neural Network]() Collection index: [llms.txt]() Data updated: 2025-10-25T21:35:37.665Z Authors: Oscar Chang, Yuling Yao, David Williams-King, Hod Lipson Published: 2019-05-23T03:58:59.000Z ## Abstract Two main obstacles preventing the widespread adoption of variational Bayesian neural networks are the high parameter overhead that makes them infeasible on large networks, and the difficulty of implementation, which can be thought of as "programming overhead." MC dropout \[Gal and Ghahramani, 2016\] is popular because it sidesteps these obstacles. Nevertheless, dropout is often harmful to model performance when used in networks with batch normalization layers \[Li et al., 2018\], which are an indispensable part of modern neural networks. We construct a general variational family for ensemble-based Bayesian neural networks that encompasses dropout as a special case. We further present two specific members of this family that work well with batch normalization layers, while retaining the benefits of low parameter and programming overhead, comparable to non-Bayesian training. Our proposed methods improve predictive accuracy and achieve almost perfect calibration on a ResNet-18 trained with ImageNet. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()