Optimal last-iterate rate for SGD in convex smooth stochastic optimization

Determine the optimal worst-case last-iterate convergence rate in expectation for Stochastic Gradient Descent applied to convex and L-smooth stochastic optimization problems without uniform gradient variance assumptions, specifically by establishing whether the best possible rate is O(1/√T) or necessarily involves an O(ln(T)/√T) factor.

Background

The paper proves an expected last-iterate bound of order O(ln(T)/√T) for SGD in convex and smooth stochastic problems without uniform variance assumptions, closing a long-standing gap in the literature. However, the authors note that the presence of the logarithmic factor may not be inherent and the exact optimal worst-case rate remains unknown. Clarifying this would settle the tightness of last-iterate bounds in the smooth convex stochastic setting and guide algorithmic design.

References

As far as we know, it is not known whether the best possible last-iterate bound for Algo:SGD under Assumption \ref{Ass:convex smooth problem} is $O(1/\sqrt{T})$ or $O(\ln(T)/\sqrt{T})$.

Algo:SGD:

$\tag{SGD} x_{t+1}=x_t-\gamma \nabla f_{i_t}(x_t), $

— Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems  (2507.14122 - Garrigos et al., 18 Jul 2025) in Remark “About the tightness of the bound,” Section 3 (Main results)

A natural next question is whether the same frontier persists for adaptive or randomized stepsize rules.

— The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives  (2609.08537 - Li et al., 8 Sep 2026) in Section 6, Conclusion