A convex combination based primal-dual algorithm with linesearch for general convex-concave saddle point problems (2401.08211v1)

Abstract: Using convex combination and linesearch techniques, we introduce a novel primal-dual algorithm for solving structured convex-concave saddle point problems with a generic smooth nonbilinear coupling term. Our adaptive linesearch strategy works under specific local smoothness conditions, allowing for potentially larger stepsizes. For an important class of structured convex optimization problems, the proposed algorithm reduces to a fully adaptive proximal gradient algorithm without linesearch, thereby representing an advancement over the golden ratio algorithm delineated in [Y. Malitsky, Math. Program. 2020]. We establish global pointwise and ergodic sublinear convergence rate of the algorithm measured by the primal-dual gap function in the general case. When the coupling term is linear in the dual variable, we measure the convergence rate by function value residual and constraint violation of an equivalent constrained optimization problem. Furthermore, an accelerated algorithm achieving the faster O(1/N^2) ergodic convergence rate is presented for the strongly convex case, where N denotes the iteration number. Our numerical experiments on quadratically constrained quadratic programming and sparse logistic regression problems indicate the new algorithm is significantly faster than the comparison algorithms.