Streaming SVD Update Models

• Streaming SVD update models are algorithmic frameworks that update singular value decompositions incrementally in real time as new data arrives.
• They employ both deterministic incremental and randomized sketch-based methods to balance computational efficiency with rigorous approximation guarantees.
• These models enable practical applications in large-scale model reduction, recommendation systems, and tensor completion by significantly reducing memory and computational costs.

A streaming SVD update model is an algorithmic framework for maintaining an approximate or exact singular value decomposition (SVD) of a data matrix whose entries, columns, or low-rank updates arrive in a sequential, streaming fashion. These methods are crucial for large-scale applications where data are too large to fit in memory simultaneously, or new data become available dynamically over time—requiring updates to the low-dimensional basis and any downstream reduced models without repeated recomputation over the entire dataset. This article surveys principal algorithms, analytical guarantees, and computational characteristics of streaming SVD update models, including incremental deterministic and randomized schemes, operator-inference integration, efficient matrix-update algorithms, sketch-based methods, and tensor extensions.

1. Incremental and Randomized Streaming SVD Algorithms

Two principal categories have emerged in streaming SVD: deterministic incremental SVD (iSVD) and randomized (sketch-based) SVD. Both aim to process each data sample or low-rank update in turn while maintaining low-memory and per-update cost.

Deterministic (Incremental) SVD: Baker’s iSVD

Given a truncated SVD $X_k = V_k \Sigma_k W_k^T$ of the sequence $X_k$ ($n \times k$, rank $r_k$), a new column $x_{k+1}$ is incorporated by projecting onto the current subspace ($q = V_k^T x_{k+1}$), forming a residual ($x_\perp = x_{k+1} - V_k q$, $p = \|x_\perp\|_2$), optionally reorthogonalizing for stability, and building the updated $(r_k + 1) \times (r_k + 1)$ “update” matrix: $$J = \begin{pmatrix} \Sigma_k & q \ 0 & p \end{pmatrix}$$ whose SVD yields $V_{k+1}, \Sigma_{k+1}, W_{k+1}$, then truncates to rank $r$ (Koike et al., 17 Jan 2026). Each update costs $O(nr)$ in time and $O(nr)$ in memory. Approximation error is controlled by spectral gaps; error can accumulate over many updates or with slowly decaying singular values.

Randomized Streaming SVD: SketchySVD

Randomized SVD accumulates lightweight sketches:

• Range sketch $Y = X \Omega^T$
• Co-range sketch $Z = \Upsilon X$
• Core sketch $C = \Xi X \Psi^T$

Each new $x_{k+1}$ updates these sketches incrementally, allowing a low-memory $O(nq + s^2)$ representation (with $q \approx 4r$, $s \approx 2q$). After streaming, a rank-$r$ SVD is extracted via a sequence of QR decompositions and a single $s \times s$ SVD, achieving expected error guarantees in Frobenius norm (Koike et al., 17 Jan 2026, Gilbert et al., 2012). Randomized approaches offer memory and computational scaling for extremely large $K$, with accuracy controlled by sketch size.

Comparison Table

Algorithm	Memory	Per-Update Time	Error Regime
Batch SVD	$O(nK)$	$O(nr)$	Truncation error
Baker’s iSVD	$O(nr)$	$O(nr)$	Accumulates w/ K
SketchySVD	$O(nq+s^2)$	$O(n\zeta)$	$O(\exp(-q/r))$

(Koike et al., 17 Jan 2026)

2. SVD-Type Matrix Update Methods for Low-Rank Changes

For data streamed as low-rank matrix increments $A_{t+1} = A_t + U_t V_t^T$, efficient updates to a bidiagonal factorization enable near-SVD-accuracy at a fraction of the cost (Brust et al., 2 Sep 2025). Two algorithms are fundamental:

Householder-type Bidiagonal Update (BHU)

BHU decouples the sparse part of the current bidiagonal $B$ from the low-rank correction $b c^T$. By representing $B + b c^T$ as $B - \bar U M^{-1} \bar V^T$ using a sequence of Householder vectors $(y_k, w_k)$ and a small triangular $M$, the updated matrix is represented via new short WY forms of Householder reflectors for both the left and right factors. Complexity per update is $O(m n^2 + n^3)$, memory $O((m+n)n)$, and the approximation error in Frobenius norm matches SVD bounds closely.

Givens-rotation Bidiagonal Update (BGU)

BGU eliminates nonzeros introduced by the low-rank update via bulge-chasing with sparse Givens rotations, each requiring $O(1)$ flops (about 10 per rotation). BGU achieves $O(n^2)$ cost and $O(n)$ extra memory, enabling high-rate updates for moderate ranks (up to thousands), with performance verified on large recommendation and network datasets (Brust et al., 2 Sep 2025).

Method	Per-update Cost	Extra Memory	Preferred Regime
BGU	$O(n^2)$	$O(n)$	High-rate, low-rank, moderate $n$
BHU	$O(m n^2 + n^3)$	$O((m+n)n)$	Rectangular, sparse-preserving, matrix-free reuse

BGU and BHU both maintain the Frobenius norm of truncated approximations to machine precision, closely matching optimal SVD methods (Brust et al., 2 Sep 2025).

3. Sketch-Based Streaming SVD and Theoretical Guarantees

In the turnstile streaming model, a sketch matrix $Y = \Phi X$ is constructed via a Johnson–Lindenstrauss (JL) transform $\Phi$, mapping $X\in\mathbb{R}^{N\times n}$ to $Y\in\mathbb{R}^{m\times n}$. Updates $(i,j,\Delta)$ to $X_{i,j}$ can be absorbed as $y_j \leftarrow y_j + \Delta \phi_i$ in $O(m)$ time per update (Gilbert et al., 2012).

For $X$ rank $k$, if $$m = O(k \epsilon^{-2} (\log(1/\epsilon) + \log(1/\delta)))$$, then:

• Singular values are preserved: $$(1-\epsilon)^{1/2} \le \sigma'_j / \sigma_j \le (1+\epsilon)^{1/2}$$
• Right singular vectors: $\|v_j - v'_j\|_2 \le \min(\sqrt{2}, \ ... \ )$ (see (Gilbert et al., 2012) for explicit expression).

Sketch-by-column streaming is thus guaranteed for spectral features as long as the sketch size and properties of $\Phi$ (e.g., subgaussian or fast-JL) hold.

4. Integration with Streaming Operator Inference

Streaming SVD underpins the Streaming Operator Inference (Streaming OpInf) paradigm for non-intrusive model reduction (Koike et al., 17 Jan 2026). The approach

1. Maintains a streaming SVD basis (Baker’s iSVD or SketchySVD) for the high-dimensional data.
2. Updates operator coefficients via recursive least-squares (RLS):

For each new projected data pair $(x̂_k, \dot x̂_k)$, with

$$d_k = [x̂_k^T, (x̂_k\otimes x̂_k)^T, u_k^T, 1] \in \mathbb{R}^{1\times d}, \quad r_k = \dot x̂_k^T \in \mathbb{R}^{1\times r}$$

the RLS update: $$c_k = 1/(1 + d_k P_{k-1} d_k^T), \ g_k = P_{k-1} d_k^T c_k, \ P_k = P_{k-1} - g_k g_k^T / c_k, \ O_k = O_{k-1} + g_k (r_k - d_k O_{k-1})$$ achieves $O(d^2)$ time and memory per step. If the SVD basis is updated, the RLS system may be restarted or reprojection can be performed using updated $W_k, \Sigma_k$ (Koike et al., 17 Jan 2026).

Streaming OpInf achieves memory reductions in excess of 99%, enables dimension reduction up to $31\,000\times$, and maintains parity with batch accuracy (Koike et al., 17 Jan 2026).

Stage	Baker’s iSVD+RLS	SketchySVD+RLS	Batch OpInf
SVD memory	$O(nr)$	$O(nq)$	$O(nK)$
LS memory	$O(d^2)$	$O(d^2)$	$O(dK)$
Total memory	$O(nr+d^2)$	$O(nq+d^2)$	$O(nK+dK)$
Final error	$\approx$ batch	$\approx$ batch	baseline

(Koike et al., 17 Jan 2026)

5. Streaming SVD for Tensor Data

Streaming SVD methodologies have been extended to tensors via the t-SVD (tensor-SVD) and related algebraic frameworks (Gilman et al., 2020). Let $$\mathcal{X} \in \mathbb{R}^{n_1 \times n_2 \times n_3}$$, with the t-product and t-SVD used to define tensor analogues of rank, basis, and projection (tubal rank, t-Grassmannian). Streaming updates are performed via incremental Grassmannian gradient descent in the block-Fourier domain.

For lateral slice $\mathcal{X}_t$ with observed entries $\Omega_t$, the per-iteration sub-problem solves a (typically small) least-squares in the FFT domain, then updates the t-Grassmannian subspace variable $\mathcal{U}_{t}$ via Riemannian gradient and retraction (frontal-slice QR in FFT domain followed by inverse FFT). The update and memory cost per timestep is $O(|\Omega_t| r d_3)$, independent of the number of slices $T$ (Gilman et al., 2020). Local expected linear convergence rates are attainable under restricted isometry and suitable initialization.

Empirically, algorithms such as TOUCAN demonstrate state-of-the-art speed and accuracy for sequential MRI/hyperspectral data, improving upon Tucker/CP-based streaming tensor trackers in both time and steady-state metrics (Gilman et al., 2020).

6. Practical Tuning and Methodological Guidelines

Key recommendations and trade-offs for deploying streaming SVD updates, especially in model reduction and operator inference contexts, include (Koike et al., 17 Jan 2026):

• Subspace dimension $r$: Choose $r$ such that $$\sum_{j>r} \sigma_j^2 \leq \epsilon \sum_j \sigma_j^2$$ with typical $\epsilon=10^{-6}$.
• iSVD truncation tolerance: Use $p < \mathrm{tol}$ with $\mathrm{tol} \approx 10^{-8} \|\Sigma_k\|_2$ for negligible loss of significance.
• Sketch sizes (SketchySVD): $q \approx 4r+1,\, s\approx 2q+1$; adjust upwards for target accuracy.
• RLS regularization: Block-diagonal $$\Gamma = \mathrm{diag}(\gamma_1 I_{r+m+1},\, \gamma_2 I_{r^2})$$; typical $\gamma \in [10^{-9}, 10^{-3}]$; tune via cross-validation.

Method selection depends critically on data geometry, update frequency, and memory/throughput constraints:

• Deterministic iSVD and BGU best suit moderate $(n, r)$ and high-frequency, low-rank update streams.
• Sketch-based and randomized SVD approaches scale to massive datasets or when streaming over $K \gg n$ columns.
• Tensor streaming methods generalize these principles to block-algebraic forms for multidimensional data.

7. Applications and Empirical Performance

Streaming SVD algorithms are central in large-scale model reduction, network analysis, recommender systems, and tensor completion, as demonstrated by:

• Streaming OpInf: Achieves $\geq 99\%$ memory reduction and order $10$–$10^3\times$ prediction speedup, with comparable model accuracy, across 1D Burgers (n=128, K~$10^5$), Kuramoto–Sivashinsky (n=512, K~$3\times 10^4$), and 3D turbulent channel flow (n~$10^7$, K=$10^4$) (Koike et al., 17 Jan 2026).
• BGU in Recommendation and Networks: BGU outperforms both LAPACK and incremental SVD, achieving sub-second update times on MovieLens 32M and benchmark suite matrices of up to $15,000 \times 6,000$, while preserving singular-value norms to $10^{-11}$ (Brust et al., 2 Sep 2025).
• Sketch SVD in Graph Laplacian Analysis: Maintains spectral guarantees for large, low-rank streaming graphs with per-update $O(m)$ time and overall $O(mn)$ memory (Gilbert et al., 2012).
• Streaming t-SVD in Tensor Completion: Attains real-time accuracy for evolving multidimensional data (tubal rank $r \approx 5$–$20$), outperforming Tucker/CP methods on hyperspectral and MRI streaming (Gilman et al., 2020).

These capacities establish streaming SVD update models as critical components for contemporary large-scale and online scientific computation.