A relaxed version of Ryu's three-operator splitting method for structured nonconvex optimization

Abstract: In this work, we propose a modification of Ryu's splitting algorithm for minimizing the sum of three functions, where two of them are convex with Lipschitz continuous gradients, and the third is an arbitrary proper closed function that is not necessarily convex. The modification is essential to facilitate the convergence analysis, particularly in establishing a sufficient descent property for an associated envelope function. This envelope, tailored to the proposed method, is an extension of the well-known Moreau envelope. Notably, the original Ryu splitting algorithm is recovered as a limiting case of our proposal. The results show that the descent property holds as long as the stepsizes remain sufficiently small. Leveraging this result, we prove global subsequential convergence to critical points of the nonconvex objective.