Subgradient Gliding Method for Nonsmooth Convex Optimization

Abstract: We identify and analyze a fundamental limitation of the classical projected subgradient method in nonsmooth convex optimization: the inevitable failure caused by the absence of valid subgradients at boundary points. We show that, under standard step sizes for both convex and strongly convex objectives, the method can fail after a single iteration with probability arbitrarily close to one, even on simple problem instances. To overcome this limitation, we propose a novel alternative termed the \textit{subgradient gliding method}, which remains well defined without boundary subgradients and avoids premature termination. Beyond resolving this foundational issue, the proposed framework encompasses the classical projected subgradient method as a special case and substantially enlarges its admissible step-size design space, providing greater flexibility for algorithmic design. We establish optimal ergodic convergence rates, $\mathcal{O}(1/\sqrt{t})$ for convex problems and $\mathcal{O}(1/t)$ for strongly convex problems, and further extend the framework to stochastic settings. Notably, our analysis does not rely on global Lipschitz continuity of the objective function, requiring only mild control on subgradient growth. Numerical experiments demonstrate that, in scenarios where the classical projected subgradient method fails completely, the proposed method converges reliably with a $100\%$ success rate and achieves orders-of-magnitude improvements in accuracy and convergence speed. These results substantially expand the scope of subgradient-based optimization methods to non-Lipschitz nonsmooth convex problems.