On a Unified and Simplified Proof for the Ergodic Convergence Rates of PPM, PDHG and ADMM

Abstract: We present a unified viewpoint of proximal point method (PPM), primal-dual hybrid gradient (PDHG) and alternating direction method of multipliers (ADMM) for solving convex-concave primal-dual problems. This viewpoint shows the equivalence of these three algorithms upto a norm change, and it leads to a four-line simple proof of their $\mathcal O(1/k)$ ergodic rates. The simple proof technique is not limited to these three algorithms, but can also be utilized to analyze related algorithms, such as gradient descent, linearized PDHG, inexact algorithms, just to name a few.