# ChannelDropBack: Forward-Consistent Stochastic Regularization for Deep Networks

Source: [ChannelDropBack: Forward-Consistent Stochastic Regularization for Deep Networks](<https://www.sota2.com/research/paper/68fa75d21e209131cea6a08f>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T03:41:42.944Z

Authors: Evgeny Hershkovitch Neiterman, Gil Ben-Artzi

Published: 2024-11-16T21:24:44.000Z

## Abstract

Incorporating stochasticity into the training process of deep convolutional networks is a widely used technique to reduce overfitting and improve regularization. Existing techniques often require modifying the architecture of the network by adding specialized layers, are effective only to specific network topologies or types of layers - linear or convolutional, and result in a trained model that is different from the deployed one. We present ChannelDropBack, a simple stochastic regularization approach that introduces randomness only into the backward information flow, leaving the forward pass intact. ChannelDropBack randomly selects a subset of channels within the network during the backpropagation step and applies weight updates only to them. As a consequence, it allows for seamless integration into the training process of any model and layers without the need to change its architecture, making it applicable to various network topologies, and the exact same network is deployed during training and inference. Experimental evaluations validate the effectiveness of our approach, demonstrating improved accuracy on popular datasets and models, including ImageNet and ViT. Code is available at \\url{https://github.com/neiterman21/ChannelDropBack.git}.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2411.10891>)

[PDF](<https://arxiv.org/pdf/2411.10891.pdf>)
