# An Adaptive Method Stabilizing Activations for Enhanced Generalization

Source: [An Adaptive Method Stabilizing Activations for Enhanced Generalization](<https://www.sota2.com/research/paper/68fa793c1e209131cea7cc74>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T04:54:52.375Z

Authors: Hyunseok Seung, Jaewoo Lee, Hyunsuk Ko

Published: 2025-06-10T02:10:40.000Z

## Abstract

We introduce AdaAct, a novel optimization algorithm that adjusts learning rates according to activation variance. Our method enhances the stability of neuron outputs by incorporating neuron-wise adaptivity during the training process, which subsequently leads to better generalization -- a complementary approach to conventional activation regularization methods. Experimental results demonstrate AdaAct's competitive performance across standard image classification benchmarks. We evaluate AdaAct on CIFAR and ImageNet, comparing it with other state-of-the-art methods. Importantly, AdaAct effectively bridges the gap between the convergence speed of Adam and the strong generalization capabilities of SGD, all while maintaining competitive execution times. Code is available at https://github.com/hseung88/adaact.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.08353>)

[PDF](<https://arxiv.org/pdf/2506.08353.pdf>)