Riemannian Multiplicative Update (RMU)

• RMU is an algorithm that fuses multiplicative update techniques with Riemannian gradient methods to enforce both nonnegativity and manifold constraints.
• It projects the Euclidean gradient onto the tangent space of a constraint manifold and employs retraction to maintain feasibility throughout optimization.
• Empirical results in chordal-NMF and sparse simplex problems show RMU improves convergence, accuracy, and constraint satisfaction over traditional methods.

Riemannian Multiplicative Update (RMU) is a class of algorithms for nonnegative matrix and tensor optimization under additional manifold constraints, synthesizing the structure-preserving characteristics of multiplicative updates with the geometric fidelity of Riemannian gradient methods. RMU is particularly relevant for optimization problems involving cone, simplex, or norm constraints (e.g., positive orthant intersections with spheres or spectrahedra), where direct Euclidean approaches inadequately reflect the true geometry of the feasible set. RMU has found application in chordal nonnegative matrix factorization (Chordal-NMF) and low-rank problems on embedded manifolds with sparse simplex constraints (Esposito et al., 2024, Esposito et al., 31 Mar 2025).

1. Mathematical Motivation and Key Geometry

Traditional multiplicative update (MU) algorithms, such as Lee–Seung's for nonnegative matrix factorization, employ elementwise ratios of negative and positive gradient components to preserve nonnegativity. However, these schemes do not enforce additional structure, such as normalization or simplex constraints, throughout the optimization process. This can result in drift in the scaling of the factors and a mismatch between the geometry of the feasible set and the optimization trajectory.

RMU instead considers the feasible set as an intersection of the nonnegative cone and a smooth Riemannian submanifold—typically a sphere (for $\ell_2$ normalization), ellipsoid, or an oblique manifold—allowing the update to respect both manifold and nonnegativity constraints. For instance, in Chordal-NMF (Esposito et al., 2024), the chordal objective

$$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$

induces a geometry (positive rays on the unit sphere) poorly matched to unconstrained Frobenius optimization, motivating RMU as a principled approach to preserve this geometry.

2. Manifold Constraints and Tangent Space Projections

The feasible set underlying RMU typically takes the form:

• The intersection of the nonnegative orthant with a (generalized) sphere, ellipsoid, or spectrahedron for Chordal-NMF (Esposito et al., 2024).
• The oblique manifold $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$, encoding $\ell_2$-normalized columns, with additional nonnegative simplex constraints encoded via a coordinate map (e.g., $H=A\odot A$) (Esposito et al., 31 Mar 2025).

For a point $x$ on such a manifold, the tangent space is defined by the local linearization of the manifold constraint (e.g., for $h$ on an ellipsoid defined by $\langle W^\top W h, h\rangle=1$, the tangent vectors $\xi$ satisfy $\langle W^\top W h, \xi\rangle=0$).

The critical RMU step involves projecting the ambient (Euclidean) gradient $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$0 onto the tangent space $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$1 of the manifold at $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$2: $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$3 where $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$4 encodes the constraint (e.g., $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$5 for an ellipsoid).

This projection ensures that the update direction is both feasible and optimal for the manifold constraint.

3. Riemannian Multiplicative Update Rule

The multiplicative update is performed not in ambient space but after projecting the (sub-)gradient onto the tangent space. The Riemannian gradient is decomposed into nonnegative parts

$$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$6

to form an elementwise rescaling ratio that automatically preserves nonnegativity: $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$7 Here, $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$8 and $$F(W,H) = \frac{1}{n}\sum_{j=1}^n \left(1 - \frac{\langle m_{:j}, Wh_{:j}\rangle}{\|Wh_{:j}\|_2}\right)$$9 denote Hadamard product and division, and $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$0 denotes a retraction operator that maps back onto the manifold (such as column $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$1 normalization for spheres/ellipsoids or the oblique manifold). The update maintains both the manifold constraint (by retraction) and nonnegativity (by elementwise positivity). For block-coordinate RMU (as in Chordal-NMF), this update is performed independently for each $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$2.

In certain contexts, such as the W-factor in Chordal-NMF, the full RMU update can be computationally expensive ($$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$3). In such cases, a projected Euclidean gradient step is preferable, preserving nonnegativity and approximately the manifold constraint with lower complexity (Esposito et al., 2024).

4. Algorithmic Implementation

For concrete problems, RMU is applied within an alternating minimization, with updates as follows:

Chordal-NMF (H-subproblem for each column $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$4):

1. Compute Euclidean gradient $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$5 for $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$6.
2. Project to the tangent space: $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$7.
3. Decompose into positive/negative parts: $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$8, $$\mathcal{OB}(r, n) = \{A \in \mathbb{R}^{r \times n}\colon \forall j, \|A_{:j}\|_2=1\}$$9.
4. Update: $\ell_2$0 (with ellipsoid retraction).

Oblique Manifold, Sparse Simplex Constraints:

Given $\ell_2$1, set $\ell_2$2. Gradient and update steps (Esposito et al., 31 Mar 2025):

1. Compute subgradient $\ell_2$3 (including $\ell_2$4- or $\ell_2$5-like sparsity penalty).
2. Project to tangent space: $\ell_2$6.
3. Decompose: $\ell_2$7, $\ell_2$8.
4. Form elementwise step: $\ell_2$9.

This process maintains all constraints by construction.

5. Convergence Properties and Computational Complexity

RMU inherits convergence guarantees from Riemannian gradient descent augmented with exact (elementwise) line searches. For smooth objectives, every limit point satisfies the Riemannian stationarity condition $H=A\odot A$0; for Lipschitz gradients, a rate of $H=A\odot A$1 or $H=A\odot A$2 (for subgradient objectives) applies (Esposito et al., 2024, Esposito et al., 31 Mar 2025). The update rule's elementwise scaling ensures nonnegativity automatically and eliminates the need for explicit feasibility projection.

Per-iteration complexity, aside from potentially expensive full-matrix updates, matches or modestly exceeds standard multiplicative update schemes: $H=A\odot A$3 for Chordal-NMF $H=A\odot A$4-block, $H=A\odot A$5 for product calculations on oblique manifolds. The retraction step (e.g., column normalization) is computationally negligible at $H=A\odot A$6. In practice, the computational cost is dominated by dense matrix-matrix products, as in traditional Euclidean methods.

6. Comparisons and Empirical Evaluation

RMU differs fundamentally from traditional Euclidean multiplicative updates by respecting the geometry of the constraint set at every iteration:

• Standard MU: $H=A\odot A$7 does not enforce normalization, requiring ad hoc renormalization that disrupts convergence in geometric settings.
• RMU: Incorporates projection onto the manifold tangent space and explicit retraction after each step, simultaneously ensuring normalization and nonnegativity.

Empirical results on synthetic and real-world NMF datasets demonstrate that RMU achieves faster objective decrease, higher solution fidelity in terms of constraint satisfaction, and superior sparsity in simplex-constrained problems relative to Euclidean schemes that only enforce constraints a posteriori (Esposito et al., 2024, Esposito et al., 31 Mar 2025). For example, in (Esposito et al., 31 Mar 2025), RMU integrated the sum-to-one constraint within the optimization flow, yielding sparsity levels (21.1% vs. 13.4% and 14.8% vs. 1.35%) unattainable via standard approaches.

7. Extensions and Outlook

RMU generalizes to arbitrary embedded manifolds for problems with additional structure, such as Stiefel or fixed-sum spheres. The method's modularity extends to blockwise updates, stochastic variants useful for large datasets, and alternative entrywise penalties (e.g., $H=A\odot A$8 for $H=A\odot A$9) (Esposito et al., 31 Mar 2025). For nonsmooth objectives or non-differentiable constraint boundaries, subgradient projection offers practical stability, though further improvements may involve smoothing, proximal, or semismooth extensions.

This suggests that RMU stands as a key methodological advance for nonnegative and simplex-constrained matrix factorization, leveraging geometric insights for improved algorithmic performance in manifold-constrained optimization.