# Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems

Source: [Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems](<https://www.sota2.com/research/paper/68fa5d601e209131ce9d985a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T18:13:08.759Z

Authors: Mohammad Partohaghighi, Roummel Marcia, YangQuan Chen

Published: 2025-03-17T22:57:37.000Z

## Abstract

Fractional-order stochastic gradient descent (FOSGD) leverages fractional exponents to capture long-memory effects in optimization. However, its utility is often limited by the difficulty of tuning and stabilizing these exponents. We propose 2SED Fractional-Order Stochastic Gradient Descent (2SEDFOSGD), which integrates the Two-Scale Effective Dimension (2SED) algorithm with FOSGD to adapt the fractional exponent in a data-driven manner. By tracking model sensitivity and effective dimensionality, 2SEDFOSGD dynamically modulates the exponent to mitigate oscillations and hasten convergence. Theoretically, this approach preserves the advantages of fractional memory without the sluggish or unstable behavior observed in na\\"ive fractional SGD. Empirical evaluations in Gaussian and $\\alpha$-stable noise scenarios using an autoregressive (AR) model\\textcolor{red}{, as well as on the MNIST and CIFAR-100 datasets for image classification,} highlight faster convergence and more robust parameter estimates compared to baseline methods, underscoring the potential of dimension-aware fractional techniques for advanced modeling and estimation tasks.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2503.13764>)

[PDF](<https://arxiv.org/pdf/2503.13764.pdf>)
