---
title: Cubic-Regularized Riemannian Newton Method
url: https://www.emergentmind.com/topics/second-order-cubic-regularized-riemannian-newton-algorithm
type: topic
---

# Cubic-Regularized Riemannian Newton Method

to=arxiv_search 񹚗json
{"query":"1805.05565 OR \"A Cubic Regularized Newton's Method over Riemannian Manifolds\"","max_results":5,"sort_by":"relevance"}
to=search_arxiv  ашәаးjson
{"query":"1805.05565","max_results":5}
The second-order cubic-regularized Riemannian Newton algorithm is a Newton-type method for minimizing a smooth function over a Riemannian manifold by solving, at each iterate, a cubic-regularized second-order model in the tangent space and then mapping the step back to the manifold by a retraction. In its canonical form, developed as the cubic regularized Riemannian Newton (CRRN) method, it addresses problems of the form
\[
\min \, f(x)\quad \text{subject to } x\in M,
\]
where \(M\) is an embedded Riemannian submanifold of a Euclidean space \(E\). Its defining feature is that it transfers Euclidean cubic-regularized Newton methodology to the manifold setting while preserving geometric feasibility and retaining the worst-case second-order complexity \(\mathcal O(1/\epsilon^{3/2})\) for reaching a second-order \(\epsilon\)-stationary point under stated smoothness and retraction assumptions [1805.05565].

## 1. Geometric problem formulation

The algorithm is designed for smooth unconstrained optimization on a manifold, but the unconstrained character is intrinsic rather than ambient: the variable is restricted to \(M\), and the ambient Euclidean problem is generally nonconvex even when the manifold itself is smooth [1805.05565]. The fundamental geometric objects are the tangent space \(T_xM\), the Riemannian gradient \(\operatorname{grad} f(x)\), the Riemannian Hessian \(\operatorname{Hess} f(x)\), and a retraction \(Retr(x,\xi)\) that maps a tangent vector \(\xi\in T_xM\) back to the manifold.

A central analytic device is the pullback of \(f\) at a base point \(x\),
\[
\hat f_x(\xi)=f(Retr(x,\xi)), \qquad \xi\in T_xM.
\]
To differentiate this object in ambient coordinates, the CRRN analysis uses an extended retraction defined by
\[
Retr(x,z)=Retr(x,P_{T_xM}z)
\]
for ambient vectors \(z\in E\), where \(P_{T_xM}\) is the orthogonal projection onto the tangent space [1805.05565]. This yields the identity
\[
\nabla_\xi \hat f_x(0)=\operatorname{grad} f(x),
\]
and a precise relation between the pullback Hessian and the Riemannian Hessian:
\[
\big\langle \nabla_{\xi}^2\hat{f}_x(0)[\eta],\eta\big\rangle
=
\big\langle Hess f(x)[\eta],\eta\big\rangle
+
\big\langle grad f(x), \tfrac{d^2}{dt^2}Retr(x,t\eta)\big|_{t=0}\big\rangle.
\]

If the retraction is second-order, the additional term vanishes, so the pullback Hessian and the Riemannian Hessian coincide on tangent directions [1805.05565]. This correspondence is structurally important because cubic regularization is formulated in the tangent space, whereas stationarity is defined on the manifold itself. When only a regular retraction is available, the analysis still controls the mismatch through
\[
\big|\big\langle(\nabla_{\xi}^2\hat{f}_x(0)-Hess f(x))[\eta],\eta\big\rangle\big|
\leq 2L_2\|grad f(x)\|\|\eta\|^2.
\]

This formulation clarifies a common misconception: the method is not simply Euclidean Newton followed by projection. Feasibility is maintained by a retraction at every iteration, and both the model and the convergence guarantees are expressed in intrinsic Riemannian terms rather than by post hoc projection arguments [1805.05565].

## 2. Cubic model and tangent-space subproblem

At iteration \(k\), the method forms a second-order Taylor model of the pullback together with a cubic regularizer. In tangent coordinates, the model is
\[
m_{x,\sigma}(\xi)
=
\hat f_x(0)
+
\langle grad f(x),\xi\rangle
+
\frac12 \langle Hess f(x)[\xi],\xi\rangle
+
\frac{\sigma}{6}\|\xi\|^3,
\qquad \xi\in T_xM.
\]
Equivalently, with \(P_x\) the orthogonal projection onto \(T_xM\),
\[
\xi_k=\arg\min_\xi \hat m_{x_k,\sigma}(\xi)
:=
\arg\min_\xi
\hat f_x(0)
+
\langle grad f(x),\xi\rangle
+
\frac12 \langle P_x\circ Hess f(x)\circ P_x[\xi],\xi\rangle
+
\frac{\sigma}{6}\|\xi\|^3.
\]
Because \(\operatorname{grad} f(x)\in T_xM\), the linear term is already tangent [1805.05565].

The subproblem is an unconstrained cubic-regularized minimization over the tangent space. In this sense, the CRRN step is the exact Riemannian analogue of Euclidean cubic Newton: the manifold enters through the tangent geometry and the retraction, while the local step computation remains a cubic-regularized second-order model minimization [1805.05565]. The iterate update is correspondingly simple:
\[
x_{k+1}=Retr(x_k,\xi_k).
\]

The global minimizer \(\xi_k\) of the cubic model satisfies Euclidean-style optimality conditions on the tangent space:
\[
(B_k+\lambda_k^* I)\xi_k + g_k = 0,\qquad
\lambda_k^*=\frac{\sigma}{2}\|\xi_k\|,\qquad
B_k+\lambda_k^* I\succeq 0,
\]
where \(g_k=grad f(x_k)\) and \(B_k=P_{x_k}Hess f(x_k)P_{x_k}\) [1805.05565]. The analysis also uses the bound
\[
\|\xi_k\|\le \frac{3}{\sigma}\max\{k_B,\sqrt{\sigma\|g_k\|}\},
\]
which keeps the step inside the region where local pullback Lipschitz estimates are valid.

The cubic term has the same role as in Euclidean cubic regularization: it stabilizes the second-order model, controls step length without an explicit trust-region radius, and enables complexity results at approximate second-order points. This places CRRN in direct contrast with Riemannian trust-region methods, whose subproblems are radius-constrained rather than cubically penalized [1805.05565].

## 3. Second-order stationarity and global complexity

The target guarantee is a second-order \(\epsilon\)-stationary point, defined by
\[
\|grad f(x^*)\|\le \epsilon,\qquad
\langle Hess f(x^*)[\xi],\xi\rangle \ge -\sqrt{\epsilon}\|\xi\|^2
\quad \forall \xi\in T_{x^*}M.
\]
This is the natural Riemannian counterpart of approximate first-order stationarity combined with approximate positive semidefiniteness of the Hessian [1805.05565].

The main global theorem for CRRN states that the method reaches such a point within \(\mathcal O(1/\epsilon^{3/2})\) iterations, provided several assumptions hold. The key requirements are that \(M\) is compact, the retraction is second-order in the global complexity proof, and the pullback Hessian is locally Lipschitz continuous uniformly on bounded tangent sets:
\[
\big|\big\langle(\nabla_{\xi}^2\hat{f}_x(\eta)-\nabla_{\xi}^2\hat{f}_x(0))[\nu],\nu\big\rangle\big|
\le L_H^R \|\eta\|,
\quad \|\eta\|\le R,\ \|\nu\|=1.
\]
Compactness is used to guarantee uniform constants and a global lower bound on \(f\), while the second-order retraction assumption ensures the cleanest transfer of pullback information to manifold optimality conditions [1805.05565].

Under a sufficiently large \(\sigma\), the method achieves a cubic decrease:
\[
f(x_k)-f(x_{k+1}) \ge \frac{\tau_1}{4}\|\xi_k\|^3,
\]
with an explicit positive \(\tau_1\) depending on \(\sigma,L_2,L_H^R,k_B,G\) [1805.05565]. Summation of these decreases yields a bound on \(\sum_k \|\xi_k\|^3\), and choosing the best iterate among \(T\) steps gives the stated complexity. The returned point \(x_{k^*+1}\) satisfies
\[
\|grad f(x_{k^*+1})\|\le \epsilon,\qquad
\langle Hess f(x_{k^*+1})[\eta],\eta\rangle\ge -\sqrt{\epsilon}\|\eta\|^2
\quad \forall \eta\in T_{x_{k^*+1}}M.
\]

A further ingredient in the proof is a Lipschitz bound on the smallest Riemannian Hessian eigenvalue,
\[
|\lambda_{\min}^M(Hess f(x))-\lambda_{\min}^M(Hess f(y))| \le D\|x-y\|,
\]
for nearby points on a compact manifold [1805.05565]. This controls the transfer of approximate second-order information between successive tangent spaces.

The complexity statement is notable because it matches the best-known Euclidean worst-case second-order complexity, and the paper explicitly contrasts it with Riemannian trust-region methods, which typically guarantee second-order \(\epsilon\)-stationarity in \(\mathcal O(1/\epsilon^{2.5})\) iterations [1805.05565]. A plausible implication is that cubic regularization is not merely a globalization device on manifolds; it is the mechanism that preserves the canonical Euclidean second-order complexity order in the intrinsic setting.

## 4. Local superlinear and quadratic convergence

Beyond worst-case guarantees, the CRRN analysis identifies two local acceleration regimes near a local minimizer \(\bar x\) [1805.05565]. The first is based on local gradient domination:
\[
f(x)-f(\bar x)\le \tau_f \|grad f(x)\|^p
\]
in a neighborhood of \(\bar x\). In that setting, the method improves on the global rate through a sharper descent recursion. The theorem introduces an auxiliary sequence \(z_k\) and shows:

- if \(p=\frac32\), then \(z_k\) decreases linearly;
- if \(1\le p<\frac32\), then
  \[
  z_k = \mathcal{O}(k^{-\beta}),\qquad \beta=\frac{2p}{3-2p},
  \]
  and in particular \(p=1\) gives \(z_k=\mathcal{O}(k^{-2})\);
- if \(p>\frac32\), then the convergence is superlinear of order \(\frac{2p}{3}\) once the iterates are sufficiently close.

The heuristic behind these regimes is explicit in the paper: the cubic model produces
\[
f(x_k)-f(x_{k+1})\gtrsim \|\xi_k\|^3,
\]
while the next gradient is controlled by
\[
\|grad f(x_{k+1})\|\lesssim \|\xi_k\|^2.
\]
Combined with gradient domination, this yields a stronger recursion than the global analysis [1805.05565].

The second regime concerns a nondegenerate local minimum. If there exists \(\delta>0\) such that
\[
\langle Hess f(\bar x)[\xi],\xi\rangle \ge \delta \|\xi\|^2
\quad \forall \xi\in T_{\bar x}M,
\]
then the method attains local quadratic convergence. Specifically, once the iterates are sufficiently close and the initial tangent step is sufficiently small,
\[
\frac{\tau_2}{\delta_0/2}\|\xi_{k+1}\|
\le
\left(\frac{\tau_2}{\delta_0/2}\|\xi_k\|\right)^2,
\]
which implies
\[
\|grad f(x_k)\|=\mathcal{O}(2^{-2^k}).
\]
The proof uses the positive definiteness of the operator
\[
Hess f(x_k)+(\sigma/2)\|\xi_k\|I
\]
together with Lipschitz continuity of the Riemannian Hessian to preserve positivity in a neighborhood [1805.05565].

These local results underscore another important distinction. The cubic regularizer does not preclude Newton-like fast local behavior; rather, when curvature is favorable, the method transitions from globally safeguarded second-order descent to locally quadratic convergence.

## 5. Retractions, compactness, and manifold-specific structure

Several assumptions in CRRN are geometric rather than purely analytic. The extended retraction is needed to differentiate pullbacks in ambient Euclidean coordinates, and second-order retractions are used to align pullback Hessians with Riemannian Hessians at the origin [1805.05565]. The regularity bounds
\[
\|Retr(x,\xi)-x\|\le L_1\|\xi\|,\qquad
\|Retr(x,\xi)-x-\xi\|\le L_2\|\xi\|^2
\]
are repeatedly invoked in the descent and neighborhood arguments.

Compactness of \(M\) is also central in the original global theory. It ensures uniform constants, a global lower bound on the objective, and local Lipschitz properties of the pullback Hessian on bounded tangent sets [1805.05565]. This compactness requirement is one of the most visible limitations of the original formulation, and later work on adaptive, inexact, and quasi-Newton variants can be read as efforts to relax or bypass aspects of that framework while preserving second-order guarantees [2405.02588].

The Stiefel specialization in the original paper makes the abstract assumptions concrete. For the Stiefel manifold \(St_{n,r}\),
\[
T_XSt_{n,r}=\{Z:\ X^\top Z+Z^\top X=0\}, \qquad
P_X(G)=G-\frac12(XX^\top G+XG^\top X),
\]
and the polar retraction
\[
Retr(X,Z)=(X+Z)(I_r+Z^\top Z)^{-1/2}
\]
is a second-order retraction with \(L_1=L_2=1\) [1805.05565]. In this case the paper provides explicit constants for the pullback Hessian Lipschitz bound and gradient comparison, including
\[
L_H = 13.66\,G + 12.55\,\ell_f + 4\,\ell_H, \qquad C_g=\frac{1}{8.62}.
\]
This explicit specialization shows that the theory is not purely formal: the regularity assumptions can be verified in an important matrix-manifold setting.

A common misunderstanding is that such constants are merely bookkeeping artifacts. In the CRRN framework they determine the admissible regularization magnitude, the validity region of pullback Taylor estimates, and the quantitative form of the descent inequality [1805.05565].

## 6. Later variants and related developments

Subsequent work has broadened the cubic-regularized Riemannian Newton paradigm in several directions while preserving the core idea of solving a cubic-regularized second-order model on tangent spaces.

| Method | Distinguishing feature | Stated complexity |
|---|---|---|
| CRRN [1805.05565] | Exact cubic-regularized Newton on the full tangent space | \(\mathcal O(1/\epsilon^{3/2})\) to a second-order \(\epsilon\)-stationary point |
| RDRSOM [2304.10092] | Cubic model solved on a low-dimensional subspace \(V_k\subset T_{x_k}M\) | \(\mathcal O(1/\epsilon^{3/2})\) for \(\epsilon\)-stationarity under Assumption 1 |
| Adaptive inexact ARC on manifolds [2405.02588] | Inexact gradient and Hessian with acceptance ratio \(\rho_k\) and adaptive \(\sigma_k\) | \(\mathcal O(\max\{\varepsilon_g^{-2},\varepsilon_H^{-3}\})\) |
| Adaptive cubic quasi-Newton [2402.12464] | Inexact \(g_{k,\alpha}\), \(\mathcal B_{k,\alpha}\), possibly derivative-free finite differences | \(\mathcal O(\epsilon_g^{-3/2})\) first-order and \(\mathcal O(\max\{\epsilon_g^{-3/2},\epsilon_H^{-3}\})\) second-order |
| Inexact Sub-RN-CR [2302.11076] | Subsampled gradient and Hessian plus cubic regularization | \(\mathcal O(\max(\epsilon_g^{-2},\epsilon_H^{-3}))\), improved to \(\mathcal O(\max(\epsilon_g^{-3/2},\epsilon_H^{-3}))\) under stronger subproblem conditions |

The dimension-reduced method RDRSOM restricts the cubic model to a subspace \(V_k\subset T_{x_k}M\), with a notable two-dimensional choice
\[
V_k = \operatorname{span}\Big(\operatorname{grad} f(x_k),\; T_{x_k}(x_k-x_{k-1})\Big),
\]
thereby reducing per-iteration cost while retaining an \(\mathcal O(1/\epsilon^{3/2})\) iteration bound under its model-accuracy assumptions [2304.10092]. Its practical Barzilai–Borwein-like extension replaces exact Hessian-vector products by finite differences and drops to \(O(\epsilon^{-2})\) complexity.

Adaptive inexact cubic regularization on manifolds allows \(G_k\approx \mathrm{grad} f(x_k)\) and \(H_k\approx \mathrm{Hess} f(x_k)\), uses the model
\[
m_k(\eta) = \langle G_k,\eta\rangle +\frac12\langle H_k[\eta],\eta\rangle +\frac13 \sigma_k \|\eta\|^3,
\]
and accepts or rejects steps through
\[
\rho_k = \frac{f(x_k)-f\circ R_{x_k}(\eta_k)}{-m_k(\eta_k)}.
\]
Its main theorem gives termination after at most
\[
\mathcal O\!\left(\max\{\varepsilon_g^{-2},\varepsilon_H^{-3}\}\right)
\]
iterations under pullback accuracy, inexactness, and subproblem-quality assumptions [2405.02588].

The adaptive cubic quasi-Newton method on manifolds replaces exact second-order information by approximations satisfying step-dependent accuracy bounds and can be implemented with forward finite differences, including a fully function-evaluation-based Hessian approximation [2402.12464]. The paper explicitly contrasts this adaptive framework with earlier Riemannian cubic methods that required compactness, constants such as \(L_2,\kappa_B,L_H^{\mathcal R},G\), and a fixed \(\sigma\) satisfying a complicated lower bound.

Subsampled cubic-regularized Riemannian Newton-type methods address large finite-sum problems by replacing exact gradient and Hessian evaluations with sampled estimates and by solving the cubic subproblem approximately with Lanczos or nonlinear conjugate gradient methods [2302.11076]. These methods aim to preserve the saddle-escape and second-order properties of cubic regularization while reducing the cost of curvature computation.

Later application-driven work has also made the paradigm more concrete. RDRSOM was applied to sensor network localization on a feasible set described as an intersection of spheres with different centers and radii, which the authors characterize as a Riemannian manifold not previously considered in the literature [2304.10092]. A more recent study formulates \(K\)-means clustering as smooth unconstrained optimization over a submanifold and states that a second-order cubic-regularized Riemannian Newton algorithm can be implemented so that each Newton subproblem is solved in linear time in the number of samples after product-manifold factorization [2509.21675].

These developments clarify the present status of the subject. The exact CRRN method is the foundational second-order cubic-regularized Riemannian Newton algorithm [1805.05565]. Many later methods are best understood not as replacements, but as structural variants: dimension-reduced, adaptive, inexact, quasi-Newton, subsampled, or application-specialized extensions of the same tangent-space cubic-regularization principle [2304.10092].

Source: https://www.emergentmind.com/topics/second-order-cubic-regularized-riemannian-newton-algorithm