# Regularizing Neural Networks by Stochastically Training Layer Ensembles

Source: [Regularizing Neural Networks by Stochastically Training Layer Ensembles](<https://www.sota2.com/research/paper/68fa66b81e209131cea12164>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T21:57:21.263Z

Authors: Alex Labach, Shahrokh Valaee

Published: 2019-11-21T18:56:29.000Z

## Abstract

Dropout and similar stochastic neural network regularization methods are often interpreted as implicitly averaging over a large ensemble of models. We propose STE (stochastically trained ensemble) layers, which enhance the averaging properties of such methods by training an ensemble of weight matrices with stochastic regularization while explicitly averaging outputs. This provides stronger regularization with no additional computational cost at test time. We show consistent improvement on various image classification tasks using standard network topologies.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1911.09669>)

[PDF](<https://arxiv.org/pdf/1911.09669.pdf>)