First-Order Projected Algorithms With the Same Linear Convergence Rate Bounds as Their Unconstrained Counterparts

Abstract: In this paper, we propose a systematic approach for constructing first-order projected algorithms for optimization problems with general set constraints, building upon their unconstrained counterparts. We show that these projected algorithms retain the same linear convergence rate bounds, when the latter are obtained for the unconstrained algorithms via quadratic Lyapunov functions arising from integral quadratic constraint (IQC) characterizations. The projected algorithms are constructed by applying a projection in the norm induced by the Lyapunov matrix, ensuring both constraint satisfaction and optimality at the fixed point. Furthermore, under a linear transformation associated with this matrix, the projection becomes non-expansive in the Euclidean norm, allowing the use of the contraction mapping theorem to establish convergence. Our results indicate that, when analyzing worst-case convergence rates or when synthesizing first-order optimization algorithms with potentially higher-order dynamics, it suffices to focus solely on the unconstrained dynamics.