# Restoring Neural Network Plasticity for Faster Transfer Learning Source: [Restoring Neural Network Plasticity for Faster Transfer Learning]() Collection index: [llms.txt]() Data updated: 2026-03-25T05:00:04.214Z Authors: Xander Coetzer, Arn\\'e Schreuder, Anna Sergeevna Bosman Published: 2026-03-21T15:41:58.000Z ## Abstract Transfer learning with models pretrained on ImageNet has become a standard practice in computer vision. Transfer learning refers to fine-tuning pretrained weights of a neural network on a downstream task, typically unrelated to ImageNet. However, pretrained weights can become saturated and may yield insignificant gradients, failing to adapt to the downstream task. This hinders the ability of the model to train effectively, and is commonly referred to as loss of neural plasticity. Loss of plasticity may prevent the model from fully adapting to the target domain, especially when the downstream dataset is atypical in nature. While this issue has been widely explored in continual learning, it remains relatively understudied in the context of transfer learning. In this work, we propose the use of a targeted weight re-initialization strategy to restore neural plasticity prior to fine-tuning. Our experiments show that both convolutional neural networks (CNNs) and vision transformers (ViTs) benefit from this approach, yielding higher test accuracy with faster convergence on several image classification benchmarks. Our method introduces negligible computational overhead and is compatible with common transfer learning pipelines. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on DTD (Accuracy Gain, Epoch Gain)](): Image Classification; DTD; Accuracy Gain: 2.4. - [Image Classification on Brain Tumor](): Image Classification; Brain Tumor; Accuracy Gain: 0.7. - [Image Classification on Fruit25](): Image Classification; Fruit25; Accuracy Gain: 90. ## Original sources and implementation [Original paper]() [PDF]()