# Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Loss Source: [Activation by Interval-wise Dropout: A Simple Way to Prevent Neural Networks from Plasticity Loss]() Collection index: [llms.txt]() Data updated: 2025-10-26T05:01:45.766Z Authors: Sangyeon Park, Isaac Han, Seungwon Oh, Kyung-Joong Kim Published: 2025-02-03T13:34:53.000Z ## Abstract Plasticity loss, a critical challenge in neural network training, limits a model's ability to adapt to new tasks or shifts in data distribution. This paper introduces AID (Activation by Interval-wise Dropout), a novel method inspired by Dropout, designed to address plasticity loss. Unlike Dropout, AID generates subnetworks by applying Dropout with different probabilities on each preactivation interval. Theoretical analysis reveals that AID regularizes the network, promoting behavior analogous to that of deep linear networks, which do not suffer from plasticity loss. We validate the effectiveness of AID in maintaining plasticity across various benchmarks, including continual learning tasks on standard image classification datasets such as CIFAR10, CIFAR100, and TinyImageNet. Furthermore, we show that AID enhances reinforcement learning performance in the Arcade Learning Environment benchmark. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()