# Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence and Experiments

Source: [Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence and Experiments](<https://www.sota2.com/research/paper/68fa5acd1e209131ce9c2e62>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T16:41:51.013Z

Authors: M. Soheil Shamaee, S. Fathi Hafshejani

Published: 2023-09-03T19:21:59.000Z

## Abstract

This paper introduces a novel approach to enhance the performance of the stochastic gradient descent (SGD) algorithm by incorporating a modified decay step size based on $\\frac{1}{\\sqrt{t}}$. The proposed step size integrates a logarithmic term, leading to the selection of smaller values in the final iterations. Our analysis establishes a convergence rate of $O(\\frac{\\ln T}{\\sqrt{T}})$ for smooth non-convex functions without the Polyak-{\\L}ojasiewicz condition. To evaluate the effectiveness of our approach, we conducted numerical experiments on image classification tasks using the FashionMNIST, and CIFAR10 datasets, and the results demonstrate significant improvements in accuracy, with enhancements of $0.5\\%$ and $1.4\\%$ observed, respectively, compared to the traditional $\\frac{1}{\\sqrt{t}}$ step size. The source code can be found at \\\\\\url{https://github.com/Shamaeem/LNSQRTStepSize}.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2309.01248>)

[PDF](<https://arxiv.org/pdf/2309.01248.pdf>)