LAI-SNMPBB: Low-Rank SNMPBB for Large-Scale SymNMF

• LAI-SNMPBB is a variant of SNMPBB that reformulates symmetric NMF using a split penalty and replaces full matrix products with a randomized low-rank surrogate.
• It employs a nonmonotone projected Barzilai–Borwein gradient approach with alternating updates, preserving curvature information for reliable step-size determination.
• Empirical evaluations on large-scale SuiteSparse matrices show improved runtime and residual quality, highlighting the tradeoff between approximation error and scalability.

LAI-SNMPBB is the large-problem-with-low-rank-approximations variant of SNMPBB, a nonmonotone projected Barzilai–Borwein method for symmetric nonnegative matrix factorization (Symmetric NMF). It is designed for large symmetric nonnegative matrices $V$, typically similarity matrices, and preserves the split-penalty SymNMF formulation of SNMPBB while replacing full matrix products by a randomized low-rank surrogate. Within the same solver family, SNMPBB is the base optimizer and Graph-SNMPBB adds graph Laplacian regularization; LAI-SNMPBB is the variant intended for large-scale inputs where the $V$-multiplications dominate runtime (Swart et al., 1 Jun 2026).

1. Position within symmetric NMF

The underlying problem is Symmetric NMF, which seeks a nonnegative factor $W \in \mathbb{R}^{n\times r}_+$ such that

$V \approx WW^T.$

The direct formulation is

$f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$

which is quartic in $W$ and nonconvex. The method family containing LAI-SNMPBB does not optimize this quartic form directly. Instead, it adopts the split penalty reformulation

$$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$

with $W\in\mathbb{R}^{n\times r}_+$, $H\in\mathbb{R}^{r\times n}_+$, and $\lambda\ge 0$. For sufficiently large $V$0, minimizing the split problem drives $V$1, so $V$2 (Swart et al., 1 Jun 2026).

This formulation supplies the common computational core for the three variants introduced in the same work. SNMPBB applies the projected Barzilai–Borwein scheme to the split objective, Graph-SNMPBB augments it by graph Laplacian regularization, and LAI-SNMPBB replaces $V$3 by a low-rank approximation while keeping the same penalty structure. The projection onto the nonnegative orthant is elementwise: $V$4

2. Projected Barzilai–Borwein mechanics

SNMPBB, and therefore LAI-SNMPBB, alternates between $V$5- and $V$6-updates and treats each block as a projected-gradient subproblem. With $V$7 fixed, the gradient is

$V$8

and with $V$9 fixed,

$W \in \mathbb{R}^{n\times r}_+$0

For one block, the method first computes a projected trial point

$W \in \mathbb{R}^{n\times r}_+$1

The paper notes that the true Lipschitz constant for $W \in \mathbb{R}^{n\times r}_+$2 is

$W \in \mathbb{R}^{n\times r}_+$3

but uses $W \in \mathbb{R}^{n\times r}_+$4 in practice because $W \in \mathbb{R}^{n\times r}_+$5 is usually small relative to $W \in \mathbb{R}^{n\times r}_+$6. From $W \in \mathbb{R}^{n\times r}_+$7, the search direction is

$W \in \mathbb{R}^{n\times r}_+$8

The Barzilai–Borwein step length is built from the Frobenius-inner-product secant fit

$W \in \mathbb{R}^{n\times r}_+$9

$V \approx WW^T.$0

and then safeguarded by

$V \approx WW^T.$1

Globalization is provided by a nonmonotone Armijo line search. Defining

$V \approx WW^T.$2

the accepted step satisfies

$V \approx WW^T.$3

and the relaxed update is

$V \approx WW^T.$4

The same scheme is applied, mutatis mutandis, to the $V \approx WW^T.$5-block. In the reported experiments, $V \approx WW^T.$6 is described as a good empirical choice, and $V \approx WW^T.$7 is typically $V \approx WW^T.$8 (Swart et al., 1 Jun 2026).

3. Low-rank approximation and the definition of LAI-SNMPBB

LAI-SNMPBB targets the principal cost bottleneck of SNMPBB: repeated multiplication by the full matrix $V \approx WW^T.$9. The large-scale variant adopts the low-rank-approximation methodology of Hayashi et al. For a random sketch matrix $f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$0, it computes

$f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$1

by a thin QR factorization, then forms

$f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$2

The factors are defined by

$f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$3

yielding the approximation

$f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$4

LAI-SNMPBB is then SNMPBB applied to the approximate objective

$f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$5

Its gradients are obtained by replacing $f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$6 by $f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$7: $f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$8

$f(W,W)=\frac{1}{2}\left\|V-WW^T\right\|^2,$9

The approximation is not formed densely in subsequent products. The paper explicitly gives the pattern

$W$0

so matrix-vector and matrix-matrix operations scale with the sketch size $W$1 rather than with $W$2. This changes the reported leading-order cost from $W$3 for SNMPBB to $W$4 for LAI-SNMPBB, which is the central algorithmic reason for introducing the variant (Swart et al., 1 Jun 2026).

4. Curvature preservation and convergence properties

A distinctive analytical result is that the low-rank approximation perturbs the gradient but does not perturb the Barzilai–Borwein curvature information within a fixed-block subproblem. Let

$W$5

For fixed $W$6,

$W$7

so the gradient bias obeys

$W$8

However, the BB update depends on gradient differences, and the constant bias cancels: $W$9 Accordingly, the BB step

$$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$0

is unchanged by the LAI approximation. The paper presents this as exact preservation of BB curvature information under randomized approximation (Swart et al., 1 Jun 2026).

The convergence guarantee for SNMPBB is global convergence to first-order stationary points of the split objective. The analysis uses blockwise convexity and Lipschitz continuity of the block gradients; for fixed $$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$1,

$$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$2

For LAI-SNMPBB, the same nonmonotone projected-gradient framework applies to the approximate objective built from $$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$3. The convergence statement therefore concerns first-order stationary points of the low-rank surrogate problem rather than of the exact full-input problem.

The paper also cites the residual-gap estimate

$$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$4

which bounds the residual deterioration incurred by solving the approximate problem. This makes the sketch quality $$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$5 the central approximation parameter: as $$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$6 decreases, the LAI solution becomes closer, in residual, to the best exact factorization (Swart et al., 1 Jun 2026).

5. Empirical evaluation

The main empirical target of LAI-SNMPBB is large-scale factorization. The paper evaluates it on 34 symmetric SuiteSparse matrices selected by a density threshold, spanning structural engineering, graph combinatorics, and biological gene networks, with sizes from $$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$7 to $$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$8. The principal comparator is LAI-SymPGNCG, identified in the paper as the best LAI method from Hayashi et al. Both methods use the same residual metric,

$$\min_{W\ge 0,\;H\ge 0} f(W,H;\lambda), \qquad f(W,H;\lambda)=\frac{1}{2}\left\|V-WH\right\|^2+\frac{\lambda}{2}\left\|W-H^T\right\|^2,$$9

and the same stopping criterion. The reported outcome is that LAI-SNMPBB outperforms LAI-SymPGNCG in both runtime and residual quality on the SuiteSparse benchmark, and the performance profile indicates that LAI-SNMPBB attains the lowest residual on about 70% of problems at performance ratio $W\in\mathbb{R}^{n\times r}_+$0 (Swart et al., 1 Jun 2026).

A denser clustering-oriented test is also reported on the Web of Science dataset with $W\in\mathbb{R}^{n\times r}_+$1 term documents and 7 categories. Averaged over ten runs, LAI-SNMPBB attains mean residual $W\in\mathbb{R}^{n\times r}_+$2 and mean adjusted Rand index $W\in\mathbb{R}^{n\times r}_+$3, while LAI-SymPGNCG attains mean residual $W\in\mathbb{R}^{n\times r}_+$4 and mean adjusted Rand index $W\in\mathbb{R}^{n\times r}_+$5. The interpretation given in the paper is therefore nuanced: LAI-SNMPBB reaches essentially comparable residuals and does so in consistently less time, but LAI-SymPGNCG has a slightly better ARI on that dataset.

The broader solver family provides context for these results. On synthetic data, SNMPBB is reported to achieve 6 times speedup over SymANLS for similar residuals, and across six real-world clustering benchmarks Graph-SNMPBB matches or exceeds SymANLS accuracy. This suggests that LAI-SNMPBB inherits its large-scale behavior from a first-order scheme that is already competitive before the low-rank-input acceleration is introduced (Swart et al., 1 Jun 2026).

6. Practical use, parameterization, and limitations

The method is intended for large symmetric nonnegative matrices where the dominant cost lies in $W\in\mathbb{R}^{n\times r}_+$6-products. A typical initialization in the paper is a random matrix with entries in $W\in\mathbb{R}^{n\times r}_+$7, scaled by

$W\in\mathbb{R}^{n\times r}_+$8

where $W\in\mathbb{R}^{n\times r}_+$9 is the average value of the entries of $H\in\mathbb{R}^{r\times n}_+$0, followed by $H\in\mathbb{R}^{r\times n}_+$1. Recommended parameter settings for the base method include

$H\in\mathbb{R}^{r\times n}_+$2

and, for Graph-SNMPBB rather than LAI-SNMPBB,

$H\in\mathbb{R}^{r\times n}_+$3

The excerpt does not provide a definitive rule for selecting the sketch dimension $H\in\mathbb{R}^{r\times n}_+$4, but it does state the governing tradeoff: increasing $H\in\mathbb{R}^{r\times n}_+$5 improves the approximation by reducing

$H\in\mathbb{R}^{r\times n}_+$6

while decreasing $H\in\mathbb{R}^{r\times n}_+$7 improves the $H\in\mathbb{R}^{r\times n}_+$8 runtime.

Several implementation observations are specific to LAI-SNMPBB. On large SuiteSparse problems, the best performance was obtained when the number of inner projected-gradient iterations was capped at roughly $H\in\mathbb{R}^{r\times n}_+$9–$\lambda\ge 0$0, rather than solving each inner subproblem to high accuracy. The paper interprets this as a form of implicit regularization: overly accurate inner solves may overfit the surrogate $\lambda\ge 0$1 rather than track the true matrix $\lambda\ge 0$2. It also notes that line search can struggle on very large sparse problems because of weak curvature, and suggests reducing the maximum number of line-search iterations in that regime (Swart et al., 1 Jun 2026).

The limitations are structural rather than incidental. LAI-SNMPBB converges to stationary points of the approximate objective, not of the exact objective. Its solution quality depends directly on the low-rank approximation error $\lambda\ge 0$3. The preserved BB curvature does not eliminate the gradient bias itself; it only shows that the secant scaling is unchanged within each fixed-block subproblem. The stopping criterion shared with LAI-SymPGNCG is mentioned but not explicitly specified in the excerpt. Finally, the Web of Science result shows that faster convergence and strong residual quality do not automatically imply uniformly better clustering quality, since ARI can still favor a competing solver on some datasets.

In that sense, LAI-SNMPBB is best understood as a scalable Symmetric NMF solver for large inputs: it retains the nonmonotone projected Barzilai–Borwein structure of SNMPBB, compresses the input matrix through randomized low-rank approximation, preserves BB curvature information exactly within each block update, and empirically improves runtime and residual quality on large matrix benchmarks while introducing the usual surrogate-accuracy tradeoff governed by $\lambda\ge 0$4 (Swart et al., 1 Jun 2026).