Cubic Regularized Newton Method for Saddle Point Models: a Global and Local Convergence Analysis

Abstract: In this paper, we propose a cubic regularized Newton (CRN) method for solving convex-concave saddle point problems (SPP). At each iteration, a cubic regularized saddle point subproblem is constructed and solved, which provides a search direction for the iterate. With properly chosen stepsizes, the method is shown to converge to the saddle point with global linear and local superlinear convergence rates, if the saddle point function is gradient Lipschitz and strongly-convex-strongly-concave. In the case that the function is merely convex-concave, we propose a homotopy continuation (or path-following) method. Under a Lipschitz-type error bound condition, we present an iteration complexity bound of $$\mathcal{O}\left(\ln \left(1/\epsilon\right)\right)$$ to reach an $\epsilon$-solution through a homotopy continuation approach, and the iteration complexity bound becomes $$\mathcal{O}\left(\left(1/\epsilon\right)<sup>{\frac{1-\theta}{\theta<sup>2}}\right)$$ under a H\"{o}lderian-type error bound condition involving a parameter $\theta$ ($0&lt;\theta&lt;1$).