Variants of alternating minimization method with sublinear rates of convergence for convex optimization (1411.7120v2)

Abstract: The alternating minimization (AM) method is a fundamental method for minimizing convex functions whose variable consists of two blocks. How to efficiently solve each subproblems when applying the AM method is the most concerned task. In this paper, we investigate this task and design two new variants of the AM method by borrowing proximal linearized techniques. The first variant is very suitable for the case where half of the subproblems are hard to be solved and the other half can be directly computed. The second variant is designed for parallel computation. Both of them are featured by simplicity at each iteration step. Theoretically, with the help of the proximal operator we first write the new as well as the existing AM variants into uniform expressions, and then prove that they enjoy sublinear rates of convergence under very minimal assumptions.