--- title: Riemannian Conjugate Subgradient Method url: https://www.emergentmind.com/topics/riemannian-conjugate-subgradient-method type: topic --- # Riemannian Conjugate Subgradient Method A Riemannian conjugate subgradient method is a first-order optimization method on a Riemannian manifold that combines nonsmooth first-order information with conjugate-direction memory. In the literature represented here, the explicit formulation of such a method appears as the **Riemannian semismooth conjugate subgradient method (RSSCSM)**, proposed for nonconvex, nonsmooth optimization on manifolds. Its defining ingredients are the Riemannian Clarke subdifferential, retraction-based iterates, vector transport of previous directions, and a search direction obtained from a convex combination of a selected Riemannian subgradient and the transported previous search direction [2509.05947]. ## 1. Problem class and formal definition The problem addressed by RSSCSM is \[ \min_{x \in \mathcal M} f(x), \] where \(\mathcal M\) is a \(d\)-dimensional, complete, connected Riemannian manifold and \(f:\mathcal M\to\mathbb R\) is generally **nonconvex** and **nonsmooth**. The analysis is developed for \[ f \in \mathcal C^1_{\mathrm{sem}(\mathcal M)}, \] that is, semismooth functions on the manifold. The class explicitly includes smooth functions, convex functions, maxima of smooth functions, and compositions of semismooth functions [2509.05947]. The nonsmooth first-order object is the **Riemannian Clarke subdifferential** \[ \partial_c f(x) := \operatorname{conv}\left\{ \lim_{k\to\infty}\operatorname{grad} f(x_k) \;\middle|\; x_k\in\Omega_f,\ x_k\to x \right\}, \] where \(\Omega_f\) is the set of differentiability points of \(f\). Any \(g\in \partial_c f(x)\) is a **Riemannian Clarke subgradient**. A point \(x\) is **Riemannian Clarke stationary** if \[ 0_x\in \partial_c f(x). \] This formulation places the method beyond the scope of classical smooth Riemannian conjugate gradient schemes, which assume the availability of \(\operatorname{grad} f(x)\) at every iterate and rely on smooth line-search conditions. In the nonsmooth setting, there is no single gradient, the usual Fletcher–Reeves, Polak–Ribière, Hestenes–Stiefel, or Dai–Yuan coefficients are not directly meaningful, and descent cannot be inferred from smooth Taylor behavior. The 2025 paper explicitly presents RSSCSM as, to the authors’ knowledge, the **first CG-type method developed for Riemannian nonsmooth optimization** [2509.05947]. ## 2. Geometric and nonsmooth ingredients The directional derivative at \(x\in \mathcal M\) in direction \(\xi\in T_x\mathcal M\) is defined by \[ f'(x;\xi) =\lim_{t\to 0^+}\frac{f(\gamma(t))-f(x)}{t}, \] for any smooth curve \(\gamma\) with \(\gamma(0)=x\) and \(\gamma'(0)=\xi\). Using a retraction \(R_x\), this is equivalently \[ f'(x;\xi)=\lim_{t\to 0^+}\frac{f(R_x(t\xi))-f(x)}{t}. \] A central notion is the **directionally active subgradient** \[ \partial f_A(x;\eta) := \{g\in \partial_c f(x)\mid \langle g,\eta\rangle_x=f'(x;\eta)\}. \] For semismooth \(f\), this set is nonempty. The method therefore does not select an arbitrary Clarke subgradient; it selects subgradients active relative to directional derivatives along specified directions [2509.05947]. The required manifold operations are the tangent space \(T_x\mathcal M\), the Riemannian metric \(\langle\cdot,\cdot\rangle_x\), a retraction \(R_x:T_x\mathcal M\to \mathcal M\), the derivative of the retraction \(\mathrm D R_x(\eta)[\xi]\), and a vector transport \(\mathcal T_\eta(\xi)\). The geometric assumptions used in the convergence theory are a positive lower bound on the injectivity radius, \[ \operatorname{Inj}(\mathcal M)\ge r>0, \] an **isometric vector transport** \[ \|\mathcal T_\eta(\xi)\|=\|\xi\|, \] and a transport parallel to \(\mathrm D R_x(t\eta)[\eta]\). The third assumption is used to transfer orthogonality from a derivative-of-retraction direction to the transported previous search direction [2509.05947]. The method is formulated on general Riemannian manifolds, not only embedded ones. In coordinates, the paper gives the conversion \[ \partial_c f(x) = [\mathrm D\varphi(x)]^{-1} \left[ G_{\varphi(x)}^{-1}\, \partial_c(f\circ \varphi^{-1})(\varphi(x)) \right], \] where \(G_{\varphi(x)}\) is the metric matrix in coordinates. This makes explicit how Euclidean Clarke subgradients of a local representation are converted into Riemannian Clarke subgradients. ## 3. Construction of the conjugate-subgradient direction The iterate update is retraction-based: \[ x_{k+1}=R_{x_k}(t_k \eta_k). \] At iteration \(k\), the line-search curve is \[ \phi_k(t):=f(R_{x_k}(t\eta_k)). \] Its one-sided directional derivatives are \[ \begin{cases} \phi_{k,+}'(t)=f'(R_{x_k}(t\eta_k);\mathrm D R_{x_k}(t\eta_k)[\eta_k]),\\[3pt] \phi_{k,-}'(t)= -f'(R_{x_k}(t\eta_k);\mathrm D R_{x_k}(t\eta_k)[-\eta_k]). \end{cases} \] After a step \(t_k\) is chosen, the method selects two directionally active subgradients at the new point: \[ (g_+)_{x_{k+1} \in \partial f_A\!\left(x_{k+1};\mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right), \] and \[ (g_-)_{x_{k+1} \in \partial f_A\!\left(x_{k+1};\mathrm D R_{x_k}(t_k\eta_k)[-\eta_k]\right). \] These satisfy \[ \left\langle (g_+)_{x_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle = \phi_{k,+}'(t_k), \] and \[ \left\langle (g_-)_{x_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle = \phi_{k,-}'(t_k). \] The selected subgradient is then taken from their convex hull: \[ \tilde g_{k+1} = \lambda_{k+1}(g_-)_{x_{k+1} +(1-\lambda_{k+1})(g_+)_{x_{k+1}, \qquad \lambda_{k+1}\in[0,1]. \] If \[ \left\langle (g_+)_{x_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle - \left\langle (g_-)_{x_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle \neq 0, \] then \[ \lambda_{k+1} = \frac{ \left\langle (g_+)_{x_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle }{ \left\langle (g_+)_{x_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle - \left\langle (g_-)_{x_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle }. \] If the denominator is zero, the method sets \[ \lambda_{k+1}=\frac12. \] By construction, \[ \left\langle \tilde g_{k+1}, \mathrm D R_{x_k}(t_k\eta_k)[\eta_k]\right\rangle=0. \] Under the transport assumption used in the paper, this implies \[ \left\langle \tilde g_{k+1}, \mathcal T_{t_k\eta_k}(\eta_k)\right\rangle =0 \] [2509.05947]. The new search direction is defined as the minimum-norm vector in the convex hull of \(-\tilde g_{k+1}\) and the transported previous direction: \[ \min_{0\le \alpha\le 1} \frac12\left\| \alpha(-\tilde g_{k+1})+(1-\alpha)\mathcal T_{t_k\eta_k}(\eta_k) \right\|^2. \] The closed-form coefficient is \[ \alpha_{k+1} = \frac{\|\mathcal T_{t_k\eta_k}(\eta_k)\|^2} {\|\tilde g_{k+1}\|^2+\|\mathcal T_{t_k\eta_k}(\eta_k)\|^2}, \] and the direction update is \[ \eta_{k+1} = -\frac{\|\mathcal T_{t_k\eta_k}(\eta_k)\|^2} {\|\tilde g_{k+1}\|^2+\|\mathcal T_{t_k\eta_k}(\eta_k)\|^2}\tilde g_{k+1} + \frac{\|\tilde g_{k+1}\|^2} {\|\tilde g_{k+1}\|^2+\|\mathcal T_{t_k\eta_k}(\eta_k)\|^2} \mathcal T_{t_k\eta_k}(\eta_k). \] The initial direction is \[ \eta_1=-\tilde g_1=-g_1,\qquad g_1\in \partial_c f(x_1). \] The paper also gives a nonsmooth Riemannian Fletcher–Reeves-type direction \[ \eta_k^{FR}= \begin{cases} -g_1,& k=1,\\[3pt] -\tilde g_k + \dfrac{\|\tilde g_k\|^2}{\|\tilde g_{k-1}\|^2}\, \mathcal T_{t_{k-1}\eta_{k-1}(\eta_{k-1}^{FR}),& k>1, \end{cases} \] and shows that, under isometric transport, \[ \|\eta_k\|^2\,\eta_k^{FR} = \eta_k\,\|\tilde g_k\|^2. \] Thus the proposed direction is a rescaled version of an FR-type nonsmooth Riemannian conjugate-gradient direction [2509.05947]. ## 4. Line search, monotonicity, and convergence theory The line search accepts \(t_k\) such that \[ \phi_k(t_k)\le \phi_k(0),\qquad \phi_{k,-}'(t_k)\le 0\le \phi_{k,+}'(t_k). \] This is the first-order optimality condition for the one-dimensional semismooth line-search model. The paper integrates an **interval reduction procedure (IRP)**. If \(\phi_{k,+}'(0)<0\), the search is performed on \(t>0\); if \(\phi_{k,-}'(0)>0\), the search is performed on \(t<0\) by applying the IRP to \(l(\tau)=\phi_k(-\tau)\); otherwise the algorithm takes the null step \(t_k=0\) [2509.05947]. For a univariate semismooth function \(l\), the IRP maintains an interval \((\underline\tau_i,\bar\tau_i)\) with trial point \(\tau_i\). It accepts \(\tau_i\) if \[ l(\tau_i)1\) in the reduction and expansion rules, and in the experiments specifies \(q=0.33\), \(\rho=2\), \(\underline\tau_1=0\), \(\tau_1=1\), \(\bar\tau_1=100\), with stopping threshold \[ \bar\tau_i-\underline\tau_i\le 10^{-6}. \] A direct consequence is monotonicity: \[ f(x_{k+1}) = \phi_k(t_k)\le \phi_k(0)=f(x_k), \] so the objective sequence is **monotonically nonincreasing**. A key identity in the convergence analysis is \[ \|\eta_{k+1}\|^2 = \frac{\|\eta_k\|^2\,\|\tilde g_{k+1}\|^2} {\|\tilde g_{k+1}\|^2+\|\eta_k\|^2}, \] equivalently \[ \frac{1}{\|\eta_{k+1}\|^2} = \frac{1}{\|\eta_k\|^2} + \frac{1}{\|\tilde g_{k+1}\|^2}. \] Hence \[ \frac{1}{\|\eta_{k+1}\|^2} = \sum_{j=1}^{k+1}\frac{1}{\|\tilde g_j\|^2}. \] When the selected subgradients are bounded above by \(L\), this implies \[ \|\eta_k\|^2\le \frac{L^2}{k}\to 0. \] The vanishing of search-direction norms is one of the central technical devices in the proof [2509.05947]. The main convergence theorem states that if \(f\in \mathcal C^1_{\mathrm{sem}(\mathcal M)}\), the initial level set \[ \mathcal L_f(x_1):=\{x\in\mathcal M\mid f(x)\le f(x_1)\} \] is bounded, Assumptions 1–3 on geometry hold, and the sequence is generated by RSSCSM, then one of two conclusions holds. If the algorithm stops at some \(x_K\), then \(x_K\) is a Riemannian Clarke stationary point. If the infinite sequence has a **unique cluster point**, then that cluster point is a Riemannian Clarke stationary point. The paper also proves a more general accumulation statement: if the algorithm does not stop and \(X_*\) denotes the set of cluster points, then \[ 0\in \operatorname{conv}\left\{\bigcup_{x_*\in X_*}\partial_c f(x_*)\right\}. \] In addition, if \(x_k\) is not Clarke stationary, then the number of null steps taken at \(x_k\) is finite [2509.05947]. ## 5. Position within the broader literature Before the appearance of RSSCSM, the literature represented here was divided between smooth Riemannian conjugate-gradient methods and nonsmooth Riemannian subgradient methods without conjugacy. The distinction is substantial: the former rely on Riemannian gradients and Wolfe-type theory, whereas the latter use subgradients, generalized directional information, or projected manifold subgradient steps [2112.02572]. | Method family | Representative papers | Distinguishing feature | |---|---|---| | Smooth Riemannian conjugate gradient | [1302.0125], [2002.01644], [2112.02572] | Retraction, transport, and \(\beta_k\)-recursions built from \(\operatorname{grad} f\) | | Riemannian subgradient without conjugacy | [2202.07821], [2305.15136] | Manifold subgradient updates, but no conjugate-direction mechanism | | Riemannian conjugate subgradient | [2509.05947] | Nonsmooth CG-type update using active subgradients and transported memory | The smooth line is exemplified by the general Riemannian conjugate-gradient framework \[ \eta_{k+1} = -\grad f(x_{k+1}) + \beta_{k+1}\, s_k\, \mathscr{T}^{(k)}(\eta_k), \] with retraction-based iterates, multiple transport mechanisms, and Riemannian analogues of FR, DY, CD, PRP, HS, and LS coefficients. That framework explicitly studies **smooth unconstrained optimization on a Riemannian manifold** and does **not** develop methods for nonsmooth objectives, Clarke subgradients, generalized gradients, or subdifferentials on manifolds [2112.02572]. The 2013 globally convergent Riemannian conjugate-gradient method introduced the **scaled vector transport** \[ \mathcal{T}^0_{\eta_x}(\xi_x) = \frac{\|\xi_x\|_x}{\|\mathcal{T}^R_{\eta_x}(\xi_x)\|_{R_x(\eta_x)}} \,\mathcal{T}^R_{\eta_x}(\xi_x), \] in order to preserve norm control in a smooth Fletcher–Reeves-type algorithm. This is directly relevant to the geometry of conjugate directions on manifolds, but it is still a smooth method and not a subgradient method [1302.0125]. The 2020 hybrid Riemannian conjugate-gradient methods likewise remain in the smooth setting, combining Dai–Yuan and Hestenes–Stiefel behavior under strong Wolfe conditions, with no subgradient or generalized-gradient component [2002.01644]. On the nonsmooth side, the 2022 projected subgradient method is formulated on a Hadamard manifold and uses \[ p_{k+1}=P_X\!\left(\exp_{p_k}\left(-t_k\frac{s_k+u_k}{\|s_k+u_k\|}\right)\right), \] but it studies optimization over a manifold of parameters defining a Riemannian metric, not a conjugate subgradient mechanism [2202.07821]. The 2023 ReSync method is a Riemannian subgradient-based algorithm on \(\mathrm{SO}(d)^n\) using tangent projection and QR-based retraction, \[ X_i^{k+1}=\operatorname{Qr}\left(X_i^k-\mu_k \widetilde \nabla_R f(X_i^k)\right), \] yet it also contains **no** conjugate coefficient, **no** vector transport, and **no** transported previous direction [2305.15136]. Accordingly, “Riemannian conjugate subgradient method” should not be conflated with earlier smooth Riemannian conjugate-gradient methods. In this literature set, RSSCSM is the point at which the smooth conjugate-gradient architecture and the nonsmooth Riemannian subgradient architecture are explicitly merged [2509.05947]. ## 6. Numerical behavior, applications, and limitations The numerical experiments for RSSCSM are conducted on the sphere \(S^n\) and the manifold of SPD matrices. On the sphere, the retraction is \[ R_x(\eta)=\frac{x+\eta}{\|x+\eta\|}, \] and on the SPD manifold the exponential map is used as retraction. Parallel transport is used as vector transport in the experiments [2509.05947]. Three classes of optimization problems are reported. The first is the maximum of multiple Rayleigh quotients on \(S^n\), \[ \min_{x\in S^n} f(x)=\max_{i=1,\dots,m}\frac12 x^\top A_i x. \] The second is the Riemannian geometric median on the sphere, \[ \min_{x\in S^n}\sum_{i=1}^m w_i\arccos(x_i^\top x). \] The third is the Riemannian center of mass on SPD, \[ \min_{X\in \mathrm{SPD}} \frac12\sum_{i=1}^m \left\| \log\!\left(X^{-1/2}A_iX^{-1/2}\right) \right\|_F^2. \] The comparisons are against **RPBM** and **REsubGM** [2509.05947]. The main empirical pattern is that RSSCSM usually wins in CPU time, especially on larger problems. The paper also reports that RSSCSM often needs **more iterations** than bundle methods or \(\varepsilon\)-subgradient methods, but each iteration is cheaper because it avoids QP subproblems. As problem size increases, the QP cost in RPBM and REsubGM dominates, and the time advantage of RSSCSM widens. For the large Rayleigh-quotient case \(n=200,m=1000\), the reported times are \(0.3190\) for RSSCSM, \(41.9263\) for RPBM-exp, \(38.7180\) for RPBM-qf, and \(0.5207\) for REsubGM [2509.05947]. The method’s strengths and limitations are both explicit. Its strengths are that it is first-order, CG-like and low-memory, avoids repeated quadratic subproblems, ensures a monotonically nonincreasing objective sequence, and comes with global convergence results under semismoothness and geometric assumptions. Its limitations are equally structural: it requires semismoothness rather than mere local Lipschitz continuity, assumes a positive injectivity-radius lower bound, assumes isometric vector transport and a specific compatibility with \(\mathrm D R\), and may require nontrivial problem-dependent procedures to compute or approximate directionally active subgradients [2509.05947]. A recurring misconception is to treat any smooth Riemannian conjugate-gradient method as a “Riemannian conjugate subgradient method.” The cited literature does not support that usage. Smooth Riemannian conjugate-gradient methods provide the transport-and-retraction backbone, while nonsmooth Riemannian subgradient methods provide manifold subdifferential and generalized directional-derivative machinery; the conjugate-subgradient method emerges only when both components are combined in a single algorithmic and analytic framework [2112.02572]. Source: https://www.emergentmind.com/topics/riemannian-conjugate-subgradient-method