Inexact Riemannian Proximal DC (iRPDC)

• The paper introduces the iRPDC framework that efficiently solves nonsmooth Riemannian difference-of-convex problems on embedded manifolds.
• It establishes finite-parameter equivalence between DC surrogates and ℓ0‐regularized or ℓ0‐constrained models, providing rigorous complexity guarantees.
• The algorithm employs structured inexact subproblem solutions with adaptive linesearch, achieving an ε-Riemannian critical point in O(ε⁻²) outer iterations.

Inexact Riemannian proximal DC (iRPDC) is an algorithmic framework for nonsmooth Riemannian difference-of-convex optimization on an embedded manifold. It addresses problems of the form
[
\min_{x\in M} F(x):=f(x)+h(x)-g(x),
]
where (M) is a Riemannian submanifold of a finite-dimensional Euclidean space, (f) is smooth, and (h,g) are convex and possibly nonsmooth. The framework was introduced together with finite-parameter equivalence results that connect specific nonsmooth DC penalties on the sphere to (\ell_0)-regularized and (\ell_0)-constrained models, and with complexity guarantees showing that an (\epsilon)-Riemannian critical point is obtained in (\mathcal{O}(\epsilon^{-2})) outer iterations; one variant attains overall (\mathcal{O}(\epsilon^{-3})) complexity, matching the best-known bound reported in the literature [2509.08561].

1. Problem class and geometric setting

The iRPDC framework is formulated on a Riemannian submanifold (M\subset E\cong\mathbb{R}^n), endowed with the Euclidean inner product (\langle\cdot,\cdot\rangle) and norm (|\cdot|). For each (x\in M), the tangent space is denoted (T_xM), and the manifold structure is accessed algorithmically through a retraction
[
Retr_x:T_xM\to M,
\qquad
Retr_x(0_x)=x,\quad \frac{d}{dt}Retr_x(t\eta)\big|_{t=0}=\eta.
]
The analysis assumes retraction regularity: there exist (\iota_1,\iota_2>0) such that
[
|Retr_x(\eta)-x|\le \iota_1|\eta|,
\qquad
|Retr_x(\eta)-x-\eta|\le \iota_2|\eta|^2.
]
Typical examples mentioned for (M) are the sphere and the Stiefel manifold, with QR-based or projection-type retractions [2509.08561].

The objective decomposition
[
F(x)=f(x)+h(x)-g(x)
]
is studied under three structural hypotheses. First, (f:E\to\mathbb{R}) is continuously differentiable, globally Lipschitz in value, and satisfies the quadratic Euclidean upper bound
[
f(y)\le f(x)+\langle \nabla f(x),y-x\rangle+\frac{L_f}{2}|y-x|^2,\quad \forall x,y\in E.
]
Second, (h,g:E\to\mathbb{R}) are convex, Lipschitz, possibly nonsmooth, with (prox_{\sigma h}) and a subgradient of (g) efficiently computable. Third, a level set of (F) is compact. The smooth part is transferred to the manifold through the Riemannian gradient
[
\mathrm{grad}\,f(x)=Proj_{T_xM}(\nabla f(x)),
]
while the convex nonsmooth components use projected subdifferentials
[
\partial_R h(x)=Proj_{T_xM}(\partial h(x)),
\qquad
\partial_R g(x)=Proj_{T_xM}(\partial g(x)).
]
This formulation targets sparse manifold-constrained models in which nonsmooth DC penalties are intended to sharpen recovery relative to purely convex nonsmooth regularization [2509.08561].

2. DC modeling and exact (\ell_0) surrogacy on the sphere

A distinctive aspect of iRPDC is its use as both an algorithmic framework and a modeling framework. On the sphere (M=\mathbb{S}^{n-1}), the associated theory establishes equivalence, in the sense of global and local minimizers, between certain nonsmooth DC formulations and their (\ell_0)-regularized or (\ell_0)-constrained counterparts [2509.08561].

For (\ell_0)-regularization, the target model is
[
\min_{x\in\mathbb{S}^{n-1}} f(x)+\sigma|x|0.
]
The DC surrogate uses the capped-(\ell_1) penalty
[
\Phi(x)=\sum_i \min{\upsilon |x_i|,1},
\qquad
\min{x\in\mathbb{S}^{n-1}} f(x)+\sigma \Phi(x),
]
which decomposes as
[
h(x)=\sigma\upsilon|x|_1,
\qquad
g(x)=\sigma\sum_i \max{\upsilon |x_i|-1,0}.
]
Two equivalence statements are given. Asymptotically, if (\upsilon_t\to\infty) and (x_t^\star) solves the capped-(\ell_1) problem, every accumulation point of ({x_t^\star}) is a global or local minimizer of the (\ell_0)-regularized problem. More strongly, if (\bar x) is a Riemannian critical point of the DC model and
[
\upsilon \ge \frac{L_f^0}{\sigma}+\sqrt{n},
]
then each coordinate of (\bar x) satisfies either (|\bar x_i|\ge 1/\upsilon) or (\bar x_i=0), and therefore (\Phi(\bar x)=|\bar x|_0). Under the same parameter condition, the two problems have the same global minimizers, and every local minimizer of the DC model is a local minimizer of the (\ell_0)-regularized model. In the terminology used in the source paper, capped-(\ell_1) is then an exact DC surrogate on the sphere [2509.08561].

For cardinality constraints, the target model is
[
\min_{x\in\mathbb{S}^{n-1}} f(x)
\quad \text{s.t.}\quad
|x|0\le k.
]
The DC penalization exploits the largest-(k)-norm
[
|x|{[k]}:=|x|{[1]}+\cdots+|x|{[k]},
]
together with the equivalence
[
|x|0\le k
\iff
|x|_1-|x|{[k]}=0.
]
The penalized model is
[
\min_{x\in\mathbb{S}^{n-1}} f(x)+\gamma(|x|1-|x|{[k]}),
]
with
[
h(x)=\gamma|x|1,
\qquad
g(x)=\gamma|x|{[k]}.
]
Again there is an asymptotic regime, (\gamma_t\to\infty), under which global minimizers of the DC model converge to global minimizers of the (\ell_0)-constrained problem. The finite-parameter result rests on a global error bound for the sparse feasible set
[
\mathcal{S}k:={x\in\mathbb{S}^{n-1}:|x|_0\le k},
]
namely
[
\mathrm{dist}(x,\mathcal{S}_k)
\le
\sqrt{2}(1+\sqrt{k/n})^{-1/2}\big(|x|_1-|x|{[k]}\big).
]
If (\bar x) is a Riemannian critical point of the DC penalty model and
[
\gamma>\frac{nL_f^0}{k},
]
then (\bar x) is (k)-sparse. Under the same inequality, the (\ell_0)-constrained problem and its DC penalization have the same global minimizers, and every local minimizer of the DC penalization is a local minimizer of the original constrained problem [2509.08561].

A common simplification is to treat these equivalence statements as general manifold facts. That is inaccurate. The finite-parameter exact-surrogacy results stated above are proved on the sphere. A plausible implication is that analogous results on other manifolds would require geometry-specific arguments rather than a direct transfer of the sphere proof.

3. Riemannian criticality and the proximal DC model

The first-order stationarity notion underlying iRPDC is explicitly Riemannian and explicitly DC. If (\hat x) is a local minimizer of
[
F(x)=f(x)+h(x)-g(x),
]
then
[
0\in \mathrm{grad}f(\hat x)+\partial_R h(\hat x)-\partial_R g(\hat x).
]
Equivalently, there exist (u\in\partial_R h(\hat x)) and (v\in\partial_R g(\hat x)) such that
[
\mathrm{grad}f(\hat x)+u-v=0
\quad\text{in }T_{\hat x}M.
]
This is the exact Riemannian criticality condition [2509.08561].

For complexity analysis, the framework uses an approximate notion. A point (x\in M) is an (\epsilon)-Riemannian critical point if there exists (y\in E) with
[
|y-x|\le \epsilon
]
such that
[
\operatorname{dist}\bigl(0,\;\mathrm{grad}f(x)+\partial_R h(y)-\partial_R g(x)\bigr)\le \epsilon.
]
The asymmetry is deliberate: the gradient and the subgradient of the concave component (-g) are evaluated at (x), while the subgradient of (h) may be evaluated at a nearby point (y). This is not a cosmetic weakening; it is built into the algorithm’s stopping mechanism and its outer-iteration proof [2509.08561].

The proximal DC step is based on a majorization of the pullback (F\circ Retr_x). For any (x\in M), any (\tilde\xi_x\in\partial g(x)), and
[
\xi_x:=Proj_{T_xM}(\tilde\xi_x),
\qquad
p_x:=\mathrm{grad}f(x)-\xi_x,
]
define
[
L_x:=2\iota_2\big(|\nabla f(x)-\tilde\xi_x|+L_h^0\big)+\iota_1^2L_f.
]
Then
[
F(Retr_x(\eta))
\le
F(x)+\langle p_x,\eta\rangle+\frac{L_x}{2}|\eta|^2+h(x+\eta)-h(x),
\qquad \eta\in T_xM.
]
Moreover,
[
L_x\le L:=2\iota_2(L_f^0+L_g^0+L_h^0)+\iota_1^2L_f.
]
This yields a quadratic upper model plus the convex term (h(x+\eta)) [2509.08561].

The exact Riemannian proximal DC algorithm, denoted RPDCA in the source paper, solves at iteration (j) the tangent-space convex subproblem
[

\eta_j^\star

\arg\min_{\eta\in T_{x_j}M}
q_j(\eta)
:=
\langle p_j,\eta\rangle+\frac{\ell_j}{2}|\eta|^2+h(x_j+\eta),
]
where (L_{\min}\le \ell_j\le L_{\max}). Exact optimality implies
[
q_j(\eta_j^\star)\le q_j(0)-\frac{\ell_j}{2}|\eta_j^\star|^2.
]
Combined with the pullback majorization, this gives sufficient decrease after retraction and backtracking. When (g\equiv 0), the exact scheme reduces to an exact ManPG-type method [2509.08561].

4. Inexactness mechanism and algorithmic structure

The iRPDC framework replaces the exact tangent minimizer (\eta_j^\star) by an approximate direction (\eta_j), but does so under a structured inexactness criterion rather than by an arbitrary residual rule. At outer iteration (j), the framework selects a curvature estimate (\ell_j\in[L_{\min},L_{\max}]), defines
[
\epsilon_j:=\min{\ell_j^{-1},1}\epsilon,
]
solves the tangent subproblem inexactly, checks whether the estimated norm of (\eta_j^\star) is below (\epsilon_j), and otherwise performs a linesearch on steps (\tau_j=s^i), (s\in(0,1)), before updating (x_{j+1}=Retr_{x_j}(\tau_j\eta_j)) [2509.08561].

The approximate direction must satisfy two conditions. The first is a model-decrease inequality:
[
q_j(\eta_j)
\le
q_j(0)-\frac{(1-\rho)\ell_j}{2}|\eta_j|^2+\mu_j+c\beta_1\ell_j\epsilon_j^2.
]
The second relates the exact and inexact step norms:
[
|\eta_j^\star|
\le
\kappa|\eta_j|+\big(\chi_j+\beta_2\epsilon_j^2\big)^{1/2},
]
with nonnegative error sequences ({\mu_j}), ({\chi_j}) controlled by the cumulative bounds
[
\sum_{t=0}^j \tau_t\mu_t\le \mu,
\qquad
\sum_{t=0}^j \chi_t\le \chi.
]
The admissible parameters satisfy
[
2(\beta_1\kappa^2+\beta_2)<1.
]
The first condition allows controlled nonmonotonicity in the local model. The second makes the stopping test implementable, because it converts information on (\eta_j) into information on the inaccessible exact minimizer (\eta_j^\star) [2509.08561].

The linesearch is integrated with this inexactness model. Under the two conditions above,
[
F(Retr_{x_j}(\tau\eta_j))
\le
F(x_j)-\frac{2-\rho-L_j\ell_j^{-1}\tau}{2}\ell_j\tau|\eta_j|^2
+\tau(\mu_j+c\beta_1\ell_j\epsilon_j^2),
\qquad \tau\in[0,1].
]
The accepted step (\tau_j=s^i) is the first one satisfying
[
F(Retr_{x_j}(\tau_j\eta_j))
\le
F(x_j)-c\tau_j\ell_j|\eta_j|^2+\tau_j(\mu_j+c\beta_1\ell_j\epsilon_j^2).
]
The number of backtracking steps is uniformly bounded, and (\tau_j) is bounded away from zero. In the language of the source paper, the linesearch adaptively captures local curvature even when subproblems are solved only approximately [2509.08561].

A notable technical feature is that the inexactness criterion is made practical through the dual of the tangent subproblem. Writing
[
T_{x_j}M={\eta\in\mathbb{R}^n:B_j^\top\eta=0},
]
where the columns of (B_j) form an orthonormal basis of (T_{x_j}M^\perp), the primal problem is recast as a Euclidean minimization with linear constraints, whose dual is
[
\min_{\lambda\in\mathbb{R}^{n-d}}
\psi_j(\lambda)
:=

\frac{1}{2\ell_j}|\lambda|^2

M_{h/\ell_j}!\left(
x_j-\frac{1}{\ell_j}(p_j+B_j\lambda)
\right)
+\frac{1}{2\ell_j}|p_j|^2.
]
The associated primal unconstrained minimizer is
[

\eta_j(\lambda)

\operatorname{prox}{h/\ell_j}!\left(
x_j-\frac{1}{\ell_j}(p_j+B_j\lambda)
\right)-x_j,
]
and its tangent projection is
[
\hat\eta_j(\lambda):=Proj{T_{x_j}M}(\eta_j(\lambda)).
]
The gradient of the dual satisfies
[
\nabla \psi_j(\lambda)=-B_j^\top \eta_j(\lambda),
]
and is (\ell_j^{-1})-Lipschitz. Small (|\nabla\psi_j(\lambda)|) simultaneously controls model decrease and the discrepancy between (\hat\eta_j(\lambda)) and (\eta_j^\star). The practical dual stopping threshold
[

\varepsilon_j

\min\left{
\frac{\rho\,\tau_{j-1}\ell_{j-1}|\eta_{j-1}|^2+2\omega_0\ell_j(j+1)^{-a}+2c\beta_1\ell_j\epsilon_j^2}{4L_h^0},
\;
\frac{4L_h^0}{\ell_j}
\right}
]
depends on previous iterates rather than on the unknown exact solution. This is the implementability mechanism emphasized in the framework [2509.08561].

5. Convergence, complexity, and algorithmic variants

The main outer-iteration result states that, under the standing assumptions and the inexactness conditions above, iRPDC terminates in
[
\mathcal{O}(\epsilon^{-2})
]
outer iterations and returns an (\epsilon)-Riemannian critical point. The proof combines sufficient decrease, bounds on (\sum_j \tau_j\ell_j|\eta_j|^2), and the implementable control of (|\eta_j^\star|). At termination, the stopping rule yields (|\eta_J^\star|\le \epsilon_J\le\epsilon), and subproblem optimality translates that bound into the (\epsilon)-Riemannian criticality condition [2509.08561].

The overall complexity depends on how each dual subproblem is solved. Three variants are developed.

The first, iRPDC-NFG, applies Nesterov’s fast gradient method to a regularized dual
[

\psi_{\delta_j}(\lambda)

\psi_j(\lambda)+\frac{\delta_j}{2}|\lambda|^2,
\qquad
\delta_j=\varepsilon_j/(4L_h^0).
]
Its subproblem complexity is
[
\mathcal{O}!\left(\varepsilon_j^{-1/2}\log(1+\varepsilon_j^{-1})\right),
]
which leads to
[
\mathcal{O}(\epsilon^{-3}\log(1/\epsilon))
]
calls to (prox_h), together with (\mathcal{O}(\epsilon^{-2})) calls to (\mathrm{grad}f) and (Retr).

The second, iRPDC-BB, uses a gradient method with Barzilai–Borwein step sizes and safeguarded backtracking directly on the dual. The reported overall bound is
[
\mathcal{O}(\epsilon^{-4})
]
proximal evaluations, with the same (\mathcal{O}(\epsilon^{-2})) outer complexity for gradient and retraction calls.

The third, iRPDC-AR, applies the accumulative regularization scheme of Lan, using accelerated gradient on a sequence of regularized dual problems. Its subproblem complexity is
[
\mathcal{O}(\varepsilon_j^{-1}),
]
giving
[
\mathcal{O}(\epsilon^{-3})
]
calls to (prox_h), while retaining (\mathcal{O}(\epsilon^{-2})) calls to (\mathrm{grad}f) and (Retr). This matches the best-known overall complexity bound cited in the paper [2509.08561].

The comparison drawn in the source paper is specific. Existing algorithms for nonsmooth Riemannian DC problems either lack provable overall complexity or require (\mathcal{O}(\epsilon^{-3})) iterations in both outer and overall complexity. By contrast, iRPDC keeps the outer iteration bound at (\mathcal{O}(\epsilon^{-2})), while one variant still reaches the best-known overall (\mathcal{O}(\epsilon^{-3})) rate. When (g\equiv 0), these methods become inexact ManPG-type algorithms with provable overall complexity, which the paper states was not previously available [2509.08561].

The framework also includes practical parameter rules. The curvature estimate (\ell_j) may be updated by a Riemannian BB-type formula, the linesearch contraction is chosen as (s\in(0,1)), and the experiments use additional termination rules based on successive-iterate differences and objective changes. Warm-starting is used when solving families of related DC problems, such as continuation in penalty parameters. These implementation choices are reported as part of the practical algorithmic design rather than as independent theoretical claims [2509.08561].

6. Sparse PCA, empirical behavior, and relation to adjacent frameworks

The principal application developed for iRPDC is sparse principal component analysis on the Stiefel manifold
[
St(n,r)={X\in\mathbb{R}^{n\times r}:X^\top X=I_r}.
]
With data (A\in\mathbb{R}^{m\times n}), the base PCA objective is
[
f(X)=-\mathrm{tr}(X^\top A^\top A X).
]
The paper considers the (\ell_0)-regularized model
[
\min_{X\in St(n,r)}
-\mathrm{tr}(X^\top A^\top A X)+\sigma|X|0,
]
the (\ell_0)-constrained model
[
\min{X\in St(n,r)}
-\mathrm{tr}(X^\top A^\top A X)
\quad\text{s.t.}\quad
|X|0\le k,
]
the standard (\ell_1)-SPCA relaxation
[
\min{X\in St(n,r)}
-\mathrm{tr}(X^\top A^\top A X)+\gamma|X|1,
]
and two DC relaxations:
[
\min{X\in St(n,r)}
-\mathrm{tr}(X^\top A^\top A X)
+\gamma\sum_{i,j}\min{\upsilon|X_{ij}|,1},
]
and
[
\min_{X\in St(n,r)}
-\mathrm{tr}(X^\top A^\top A X)
+\gamma(|X|1-|X|{[k]}).
]
Both are direct instances of the (f+h-g) structure [2509.08561].

The reported experiments use synthetic data with (m=500,n=4000) and a CIFAR-10 subset from LIBSVM with (m=500,n=3072). Performance is measured by scaled variance relative to the PCA baseline and by sparsity level. The numerical findings are consistent across the two DC models. For capped-(\ell_1)-SPCA, increasing the slope parameter (\upsilon) improves the (\ell_0)-regularized objective monotonically until stabilization, and at the same sparsity level capped-(\ell_1) yields substantially higher explained variance than (\ell_1)-SPCA. One reported random-data example with (r=20) gives (\mathrm{s}p\approx 0.8) and (v{sc}\approx 0.7563) for capped-(\ell_1), versus (v_{sc}\approx 0.6167) for (\ell_1)-SPCA tuned to similar sparsity. For the (\ell_1-\ell_{[k]}) model, sufficiently large (\gamma) yields solutions that satisfy the target sparsity level (|X|_0\le k) with high variance, whereas (\ell_1)-SPCA with large (\gamma) tends to degenerate toward nearly one-hot columns with very low variance. Computationally, iRPDC variants, especially iRPDC-BB and iRPDC-ASSN, are compared with OADMM; iRPDC-BB often has the best runtime, all iRPDC algorithms deliver better objective values than OADMM at comparable sparsity, and OADMM is reported as sensitive to its penalty parameter while iRPDC is more robust [2509.08561].

Within the broader literature, iRPDC occupies a specific position. The paper contrasts it with Riemannian nonsmooth methods that handle composite structures such as (f(x)+h(\mathcal{A}x)) but do not explicitly address DC terms (h-g), and with existing Riemannian DC methods that often rely on Hadamard-manifold assumptions and therefore do not cover spheres or Stiefel manifolds [2509.08561]. This distinction is visible in nearby work. The inexact manifold proximal linear algorithm IManPL studies composite problems (f+h(c(z))), uses low- and high-accuracy stopping conditions, and establishes (\mathcal{O}(1/\epsilon^2)) outer complexity together with essentially (\mathcal{O}(1/\epsilon^3)) total inner complexity, but its analysis is organized around composite weak convexity rather than explicit DC decomposition [2508.19234]. The inexact Riemannian proximal gradient method IRPG treats (f+g) on manifolds, provides global convergence and KL-based local rates under inexact proximal solves, and supplies practical residual rules, yet it is not a DC framework in the sense of (f+h-g) modeling [2111.07269]. FRIDA develops exact and inexact proximal DC algorithms for signed Fréchet regression on manifolds with bounded curvature, using local strongly convex normal balls, curvature-dependent proximal parameters, sublinear step-size complexity, and KL-type local rates [2605.23097]. Other adjacent inexact frameworks include enlargement-based proximal methods for monotone vector fields on Hadamard manifolds [1511.08301], tangential block majorization-minimization with (\mathcal{O}(\epsilon^{-2})) complexity under summable subproblem gaps [2405.03073], and proximal gradient with an inexact first-order oracle, global stationarity, KL convergence, and explicit rates under oracle decay conditions [2606.25764].

This positioning clarifies two points. First, iRPDC is not merely an inexact manifold proximal-gradient scheme with a convex regularizer; its defining object is the explicit DC structure (f+h-g), together with a criticality notion and stopping rule adapted to that structure. Second, its sphere-based (\ell_0)-equivalence results and its (\mathcal{O}(\epsilon^{-2})) outer complexity claim are separate contributions: one concerns model fidelity, the other algorithmic complexity. The paper presents them as complementary ingredients in a unified approach to nonsmooth sparse optimization on manifolds [2509.08561].