Convergence of stochastic AGD under weak growth or stochastic line-search

Establish convergence guarantees for stochastic Nesterov accelerated gradient descent (stochastic AGD) when only relaxed assumptions hold—specifically, prove convergence under the weak growth condition (e.g., E[||∇f_i(w)||^2] ≤ 2αL(f(w) − f(w*)) for some α) or when step sizes are chosen via a stochastic line-search—in contrast to the strong growth condition setting analyzed in this work.

Background

This paper develops accelerated convergence guarantees for a generalized stochastic Nesterov accelerated gradient method under interpolation and strong growth conditions, improving prior dependence on the strong growth constant from ρ to √ρ.

The key analytical tool is an expected per-iteration progress condition that is satisfied by SGD under strong growth. However, whether analogous guarantees can be obtained under weaker noise assumptions or with stochastic line-search mechanisms is not established here, and the authors explicitly identify the lack of a proof of convergence for stochastic AGD in these relaxed settings.

References

For example, the convergence of stochastic AGD under relaxed conditions, like weak growth or with a stochastic line-search \citep{vaswani2019fast}, has not been proved.

— Faster Convergence of Stochastic Accelerated Gradient Descent under Interpolation  (2404.02378 - Mishkin et al., 2024) in Conclusion, final paragraph

Several questions remain open. The uniformly bounded stochastic-gradient noise assumption enables exact convergence with a fixed clipping threshold, but is stronger than commonly used bounded-variance conditions. It would therefore be interesting to determine whether the analysis can be extended to weaker stochastic-gradient assumptions, possibly by imposing alternative structural conditions on the clipping bias.

— Almost Sure Convergence Analysis of Stochastic Gradient Methods with Clipping and Additive Noise  (2609.12119 - Mukherjee et al., 10 Sep 2026) in Section Conclusion