Convergence of stochastic AGD under weak growth or stochastic line-search

Establish convergence guarantees for stochastic Nesterov accelerated gradient descent (stochastic AGD) when only relaxed assumptions hold—specifically, prove convergence under the weak growth condition (e.g., E[||∇f_i(w)||^2] ≤ 2αL(f(w) − f(w*)) for some α) or when step sizes are chosen via a stochastic line-search—in contrast to the strong growth condition setting analyzed in this work.

Background

This paper develops accelerated convergence guarantees for a generalized stochastic Nesterov accelerated gradient method under interpolation and strong growth conditions, improving prior dependence on the strong growth constant from ρ to √ρ.

The key analytical tool is an expected per-iteration progress condition that is satisfied by SGD under strong growth. However, whether analogous guarantees can be obtained under weaker noise assumptions or with stochastic line-search mechanisms is not established here, and the authors explicitly identify the lack of a proof of convergence for stochastic AGD in these relaxed settings.