Projection-Based Riemannian Bregman Gradient Method

• The paper introduces a projection-based method that solves an unconstrained Bregman subproblem and then projects the solution back to the manifold using a backtracking rule.
• The method leverages a quartic reference function to obtain closed-form or scalar root-finding updates, simplifying the computational effort compared to retraction-based schemes.
• The convergence theory employs projection inequalities, normal corrections, and a descent lemma to guarantee that every limit point is Riemannian-stationary with an O(1/ε²) iteration bound.

Projection-Based Riemannian Bregman Gradient Method denotes a Riemannian first-order scheme for smooth optimization over embedded submanifolds $M\subset \mathbb R^n$ when the objective $f$ is relatively smooth in the ambient Euclidean space with respect to a strongly convex reference function $h$. In "On Relatively Smooth Optimization over Riemannian Manifolds" (He et al., 5 Aug 2025), it appears as one of two Riemannian first-order methods obtained by incorporating the Bregman distance into the update steps. The projection-based variant addresses the smooth case $g\equiv 0$, solves an unconstrained subproblem in the ambient Euclidean space, and then projects the trial point back to the manifold by a backtracking rule that enforces sufficient decrease.

1. Geometric and analytic setting

The method is formulated on a $d$-dimensional $C^\infty$ embedded submanifold $M\subset \mathbb R^n$. For each $x\in M$, the tangent space is

$$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$

and the normal space is $N_xM=(T_xM)^\perp$.

The ambient objective $f$0 is assumed to be relatively smooth with respect to a reference function $f$1 that is $f$2 and $f$3-strongly convex for some $f$4. Relative $f$5-smoothness means

$f$6

where the Bregman distance is

$f$7

The optimization problem for the projection-based method is the smooth constrained problem

$f$8

Its Riemannian gradient is the tangent projection of the ambient Euclidean gradient,

$f$9

with $h$0 and $h$1 denoting orthogonal projection onto the tangent and normal spaces, respectively. This setup is intended for optimization over Riemannian embedded submanifolds when the ambient objective may fail to satisfy ordinary Lipschitz-gradient assumptions but does satisfy relative smoothness.

2. Update rule and projection mechanism

At an iterate $h$2, with stepsize parameter $h$3, the method computes a search direction by solving the unconstrained ambient-space subproblem

$h$4

The direction is then decomposed as $h$5, where $h$6 and $h$7.

A normal-space “correction” $h$8 is introduced, subject to

$h$9

with $g\equiv 0$0 given as an example. Starting from $g\equiv 0$1, the step length is reduced by a factor $g\equiv 0$2 until the sufficient-decrease condition

$g\equiv 0$3

is satisfied. The accepted update is

$g\equiv 0$4

In algorithmic form, the inputs are $g\equiv 0$5, $g\equiv 0$6, $g\equiv 0$7, and $g\equiv 0$8. Each iteration solves the Bregman model in $g\equiv 0$9, chooses a normal correction, performs backtracking, and sets $d$0 by projection. The defining structural feature is that the subproblem is unconstrained in the ambient space, whereas the manifold geometry enters through $d$1, the normal correction, and the projection $d$2.

This suggests that the method should not be identified with ordinary Euclidean projected gradient descent. The trial direction is not the raw Euclidean gradient; it is the minimizer of a Bregman model driven by the Riemannian gradient $d$3, and the projection step is only one component of the full update mechanism.

3. Explicit quartic reference-function case

A central special case arises when the reference function has the quartic form

$d$4

For this choice, the constraint subproblem of the retraction-based method admits a closed-form solution, and the projection-based method also obtains an explicit update characterization (He et al., 5 Aug 2025).

Define

$d$5

Then the first-order condition yields

$d$6

where $d$7 solves the scalar cubic

$d$8

By Cardano’s formula, $d$9 has a closed-form expression.

The paper emphasizes that the projection-based subproblem is often easier than the retraction-based tangent-space subproblem. In particular, it is stated to often admit closed form for quartic $C^\infty$0, or simple scalar root-finding in settings such as the sphere with log-barrier or entropy reference functions. A plausible implication is that the geometry carried by $C^\infty$1 can be chosen to match the growth of the ambient objective while preserving computationally tractable update rules.

4. Convergence theory

The deterministic convergence analysis assumes that $C^\infty$2 is a complete embedded $C^\infty$3 manifold, that $C^\infty$4 is relatively $C^\infty$5-smooth with respect to a $C^\infty$6-strongly convex $C^\infty$7, and that the sublevel sets of $C^\infty$8 are compact. Under these assumptions, Theorem 4.2 states that if $C^\infty$9, then every limit point of $M\subset \mathbb R^n$0 is Riemannian-stationary, meaning $M\subset \mathbb R^n$1, and reaching an $M\subset \mathbb R^n$2-approximate stationary point in the sense $M\subset \mathbb R^n$3 requires at most $M\subset \mathbb R^n$4 iterations (He et al., 5 Aug 2025).

The proof is organized around several geometric and analytic lemmas on compact sublevels:

• Projection inequalities: there exist constants $M\subset \mathbb R^n$5 and a radius $M\subset \mathbb R^n$6 such that for $M\subset \mathbb R^n$7, $M\subset \mathbb R^n$8, and $M\subset \mathbb R^n$9 with $x\in M$0,

$x\in M$1

$x\in M$2

• Normal-component bound: for nearby $x\in M$3,

$x\in M$4

• Bregman-deviation estimate: for sufficiently small accepted $x\in M$5,

$x\in M$6

where $x\in M$7 depends on $x\in M$8 and $x\in M$9 on a compact neighborhood.

• Descent lemma: if $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$0 and $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$1 is sufficiently small, with a positive lower bound, then

$$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$2

The proof sketch combines relative smoothness,

$$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$3

with a decomposition of $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$4 into the projection error and the main term $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$5. The projection inequalities and Bregman-deviation bound control the discrepancy between the ambient model step and the projected manifold step. The optimality condition for the subproblem, together with $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$6-strong convexity of $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$7, provides a negative quadratic term of order $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$8. After choosing $$T_xM=\{\gamma'(0):\gamma:(-\varepsilon,\varepsilon)\to M \text{ smooth},\ \gamma(0)=x\},$$9 small enough that this term dominates the higher-order remainders, summation yields the $N_xM=(T_xM)^\perp$0 iteration bound.

5. Stochastic extensions and comparison with related first-order methods

When the manifold is compact, the framework is extended to stochastic settings by replacing $N_xM=(T_xM)^\perp$1 with a minibatch gradient $N_xM=(T_xM)^\perp$2 satisfying

$N_xM=(T_xM)^\perp$3

The projection-based and retraction-based schemes are then applied with fixed stepsize $N_xM=(T_xM)^\perp$4 and batch size $N_xM=(T_xM)^\perp$5. The resulting estimate is

$N_xM=(T_xM)^\perp$6

and the parameter choice $N_xM=(T_xM)^\perp$7, $N_xM=(T_xM)^\perp$8 gives sample complexity $N_xM=(T_xM)^\perp$9 (He et al., 5 Aug 2025).

The source also places the method relative to the retraction-based Bregman scheme and to classical Riemannian gradient descent.

Method	Subproblem	Stated properties
Projection-based P-RBGD	Unconstrained in $f$00	Smooth Riemannian optimization; often closed-form for quartic $f$01; $f$02
Retraction-based R-Bregman	Convex subproblem in $f$03	Can handle nonsmooth $f$04; constrained and sometimes harder
Classical Riemannian GD	Squared-Euclid $f$05	Bregman versions can exploit non-Lipschitz but relatively smooth $f$06

The comparison is not only formal. The retraction-based method is stated to handle nonsmooth optimization, but its tangent-space subproblem is constrained and sometimes harder. By contrast, the projection-based method is stated to be typically simpler per iteration. Compared with classical Riemannian gradient descent, the Bregman variants can exploit objectives that are non-Lipschitz in the ordinary Euclidean-gradient sense but relatively smooth with respect to an appropriate reference function, with $f$07 given as an example.

6. Numerical illustrations

Two numerical examples are reported in support of the method’s practical behavior (He et al., 5 Aug 2025). The first is the low-rank quadratic sensing problem on the fixed-rank manifold. Measurements are generated as

$f$08

and the goal is to recover $f$09. In this setting the choice

$f$10

yields a closed-form update direction. Figures 1–4 plot $f$11 versus iteration for various $f$12 and rank $f$13, and the projection-based method is reported to converge in roughly half the iterations of Riemannian steepest descent, specifically Manopt’s “RSD” with line search.

The second example is the nonlinear eigenvalue problem

$f$14

The reported comparison includes retraction-based Bregman (R-RBGD), projection-based Bregman without correction (P-RBGD), projection-based Bregman with correction (P-RBGD-C), and Manopt’s RSD and RSD-Ada. Tables varying $f$15 and $f$16 report final function value, $f$17, iterations, and CPU time. The summary given in the source is that the Bregman methods reach $f$18 in approximately $f$19 of the iterations of RSD and RSD-Ada. CPU time is described as comparable or smaller for moderate sizes, while for large $f$20 the classical RSD stepsize may shrink too much and fail.

The paper states that these experiments demonstrate the advantages of the proposed methods, and that the results confirm both theoretical advantages and practical performance benefits of the projection-based Riemannian Bregman gradient method. A cautious interpretation is that the empirical evidence is aligned with the claimed complexity improvements in regimes where relative smoothness and non-Euclidean reference geometry are natural modeling choices.

7. Related projection-based Bregman dynamics on manifolds

A distinct but related line of work studies a projection-based Riemannian “Bregman-momentum” method through continuous-time variational dynamics and symplectic discretization rather than through first-order relative-smoothness subproblems. In "Accelerated Optimization on Riemannian Manifolds via Projected Variational Integrators" (Duruisseaux et al., 2022), the starting point is a Bregman Lagrangian on a complete Riemannian manifold $f$21 for geodesically convex, or $f$22-weakly-quasi-convex, objectives with unique minimizer $f$23, together with a strictly convex $f$24 kernel $f$25 satisfying $f$26. The corresponding Euler–Lagrange equation yields a second-order ODE, which is then converted into a time-dependent Hamiltonian system on $f$27.

The discretization in that work is time-adaptive and symplectic. It re-parameterizes the Hamiltonian flow into an autonomous Hamiltonian in an extended phase space, applies a symplectic integrator such as symplectic Euler or Stormer–Verlet, and after each explicit step performs the ambient projection

$f$28

Under geodesic $f$29-convexity, an $f$30-Lipschitz gradient condition, and bounded sectional curvature, the stated rate is

$f$31

This suggests that the phrase “projection-based Riemannian Bregman” now covers at least two technically different families. One family, exemplified by P-RBGD, is a first-order method for relatively smooth optimization over embedded submanifolds and is analyzed through stationary-point complexity $f$32. The other is a variational-integrator momentum construction analyzed in geodesically convex settings through asymptotic rates of the form $f$33. Their common feature is the use of Bregman geometry together with ambient-space projection, but their mathematical starting points, update laws, and guarantees are different.