# Dynamic gain neuromodulation attenuates the stability gap under joint training

Source: [Dynamic gain neuromodulation attenuates the stability gap under joint training](<https://www.sota2.com/research/paper/68fa5f7c1e209131ce9ea395>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-08-12T05:00:21.403Z

Authors: Alejandro Rodriguez-Garcia, Anindya Ghosh, Srikanth Ramaswamy

Published: 2025-07-18T16:34:06.000Z

## Abstract

Recent work in continual learning has highlighted the stability gap -- a temporary performance drop on previously learned tasks when new ones are introduced. This phenomenon reflects a mismatch between rapid adaptation and strong retention at task boundaries, underscoring the need for optimization mechanisms that balance plasticity and stability over abrupt distribution changes. While optimizers such as momentum-SGD and Adam introduce implicit multi-timescale behavior, they still exhibit pronounced stability gaps. Importantly, these gaps persist even under ideal joint training, making it crucial to study them in this setting to isolate their causes from other sources of forgetting. Motivated by how noradrenergic (neuromodulatory) bursts transiently increase neuronal gain under uncertainty, we introduce a dynamic gain scaling mechanism as a two-timescale optimization technique that balances adaptation and retention by transiently increasing the effective update magnitude while dynamically reparameterizing the weights governing the forward pass, thereby empirically mitigating transition-induced curvature amplification. Across domain- and class-incremental MNIST, CIFAR, and mini-ImageNet benchmarks under task-agnostic joint training, dynamic gain scaling effectively attenuates stability gaps while maintaining competitive accuracy, improving robustness at task transitions.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2507.14056>)

[PDF](<https://arxiv.org/pdf/2507.14056.pdf>)
