# Stacey: Promoting Stochastic Steepest Descent via Accelerated $\\ell\_p$-Smooth Nonconvex Optimization Source: [Stacey: Promoting Stochastic Steepest Descent via Accelerated $\\ell\_p$-Smooth Nonconvex Optimization]() Collection index: [llms.txt]() Data updated: 2025-10-27T06:54:52.186Z Authors: Xinyu Luo, Cedar Site Bai, Bolian Li, Petros Drineas, Ruqi Zhang, Brian Bullins Published: 2025-06-07T00:47:07.000Z ## Abstract While popular optimization methods such as SGD, AdamW, and Lion depend on steepest descent updates in either $\\ell\_2$ or $\\ell\_\\infty$ norms, there remains a critical gap in handling the non-Euclidean structure observed in modern deep networks training. In this work, we address this need by introducing a new accelerated $\\ell\_p$ steepest descent algorithm, called Stacey, which uses interpolated primal-dual iterate sequences to effectively navigate non-Euclidean smooth optimization tasks. In addition to providing novel theoretical guarantees for the foundations of our algorithm, we empirically compare our approach against these popular methods on tasks including image classification and language model (LLM) pretraining, demonstrating both faster convergence and higher final accuracy. We further evaluate different values of $p$ across various models and datasets, underscoring the importance and efficiency of non-Euclidean approaches over standard Euclidean methods. Code can be found at https://github.com/xinyuluo8561/Stacey . ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()