---
title: Diagonal-Plus-Low-Rank (DPLR) Updates
url: https://www.emergentmind.com/topics/diagonal-plus-low-rank-dplr-updates
type: topic
---

# Diagonal-Plus-Low-Rank (DPLR) Updates

A diagonal-plus-low-rank (DPLR) structure refers to matrices or updates of the form \( M = D + U V^T \), where \( D \) is a diagonal matrix and \( U,V \) are rectangular matrices with a small number of columns, resulting in a low-rank modification. This decomposition exploits the complementary properties of diagonal and low-rank components, unifying computational efficiency with model flexibility. DPLR updates appear across covariance estimation, matrix function evaluation, differential equations, efficient matrix factorizations, optimization, and large-scale learning settings. Their algorithmic utility is enabled by fast closed-form manipulation, efficient low-storage updates, and structural preservation through key mathematical identities.

## 1. Structural Formulation and Theoretical Properties

The canonical DPLR form is \( M = D + U V^T \), with \( D \in \mathbb{R}^{n \times n} \) diagonal and \( U, V \in \mathbb{R}^{n \times r} \), typically with \( r \ll n \) ensuring low rank. For symmetric or positive semidefinite settings, \( V = U \) yields \( M = D + U U^T \). The structure is preserved under addition of rank-\( k \) modifications, through the identity \( (D + U U^T) + X X^T = D + [U, X][U, X]^T \), or by appropriate downdating formulas [2002.08831]. Inverse computations leverage the Woodbury identity:
\[
(D + U U^T)^{-1} = D^{-1} - D^{-1}U (I + U^T D^{-1} U)^{-1} U^T D^{-1}
\]
enabling \( O(n r^2) \) cost for inversion and matrix-vector products [2407.03373]. In high-dimensional positive definite cases, DPLR matrices are always full-rank if \( D \succ 0 \), and support efficient updates of LDL factorization and related decompositions [2002.08831].

## 2. Algorithms for DPLR Updates and Optimization

DPLR structures admit specialized algorithms in both static and streaming contexts:

- **Covariance/Precision Matrix Estimation:** The DPLR estimator for the precision matrix \( \Theta = -L + D \), constrains \( L \succeq 0 \) with \(\mathrm{rank}(L)\le r\) and \( D \) diagonal, leading to the estimator via
  \[
  \arg\min_{\Theta=-L+D} \left\{ \mathrm{tr}(\Theta S) - \log|\Theta| + \tau(\mathrm{rank}(L)) \right\}
  \]
  solved by blockwise coordinate-descent updates: fixing \( D \), update \( L \) via eigen-decomposition; fixing \( L \), update \( D \) by solving a log-det SDP. This achieves statistical consistency at minimax rates and supports both fixed-rank and penalized-rank selection strategies [1802.06048].

- **Streaming Covariance Updates and LDL Factorization:** Sequential rank-\( k \) updates or downdates to \( M = D + U U^T \) employ the identities
  \[
  M_{\text{new}} = D + [U, X][U, X]^T, \quad \text{or} \quad M_{\text{new}} = D + U U^T - X X^T
  \]
  Together with direct in-place LDL factor modification algorithms, these enable \( \Theta((k+1)n^2) \) update cost, a substantial reduction over the naïve full recomputation [2002.08831]. This facilitates efficient management of covariance matrices in Bayesian online and sliding window contexts.

- **Dynamics on DPLR Manifolds:** To constrain a matrix ODE \( \dot{X} = F(X) \) to the DPLR manifold, the time derivative is projected onto the tangent space:
  \[
  \delta X = \delta D + \delta U U^T + U \delta U^T
  \]
  An efficient orthogonal projection yields vector field updates, enabling fast Riccati-type ODEs, Wasserstein flow, and Kalman filtering for high-dimensional systems, all at linear or near-linear storage and computation [2407.03373].

## 3. Efficient Matrix Function Updates in DPLR Contexts

When a matrix function \( f(M) \) must be evaluated after a DPLR modification, specialized Krylov subspace methods yield low computational cost:

- **Polynomial and Rational Krylov Updates:** For \( f(D + U V^T) - f(D) \), block Krylov subspace projections provide low-rank compressed approximations with superlinear convergence:
  \[
  f(D+ U V^T) - f(D) \approx U_m X_m(f) V_m^T
  \]
  Orthonormal bases \( U_m, V_m \) are built from repeated application of \( D \) or \( D^* \) to \( U,V \), exploiting the diagonal structure for cheap multiplications [1707.03045, 2008.11501]. For analytic \( f \), convergence is dictated by the best polynomial or rational approximation error on the spectral domain, with exponential error decay for exponential and Markov functions.

- **Matrix Square Roots and Inverse Roots:** Updates to \( (A+X X^T)^{1/2} \) or \( (A+X X^T)^{-1/2} \) are formulated via Riccati equations for the correction term \( E \), solved by low-rank Riccati solvers. Eigenvalue decay of the correction is geometric in the number of columns of \( X \), permitting accurate low-rank representations [2201.13156].

- **Sylvester Equations and ParaDiag Matrix Equations:** DPLR modifications in the context of discretized PDEs and all-at-once methods are handled through Sherman–Morrison–Woodbury identities and tensor-Krylov corrections, or by interpolation in the update parameter, enabling rapid solution of perturbed matrix equations and preconditioning strategies [2204.03073].

## 4. Structured Matrix Factorization and Spectral Algorithms

DPLR matrices facilitate fast factorization methods by exploiting quasiseparable structure:

- **Hessenberg Reduction:** For \( A = D + U V^* \), the matrix is \( k \)-quasiseparable (each strictly lower/upper triangular block has rank ≤\( k \)), allowing a Hessenberg reduction in \( O(n^2 k) \) time. At each step, the Givens-vector (GV) representation and generator matrices are updated efficiently, and the structure is preserved through nested block updates [1501.07812]. This yields backward-stable reductions and underpins fast eigenvalue solvers and determinant evaluations in polynomial eigenproblems.

| Problem              | Classical Cost   | DPLR-aware Cost    |
|----------------------|-----------------|--------------------|
| Matrix inversion     | \(O(n^3)\)      | \(O(n r^2)\)       |
| Hessenberg reduction | \(O(n^3)\)      | \(O(n^2 k)\)       |
| Krylov function eval | \(O(n^3)\)      | \(O(n m k)\)       |

## 5. Practical Applications and Statistical Performance

DPLR updates are applied across diverse domains:

- **High-dimensional Covariance Estimation:** DPLR estimators outperform pure sparse or diagonal methods in Kullback-Leibler loss for models with low-rank plus diagonal structure, and yield improved Sharpe ratios in Markowitz portfolio optimization when plugged into empirical finance pipelines [1802.06048].

- **Online and Streaming Learning:** DPLR-based LDL-updates allow efficient, scalable computation for dynamic datasets, relevant in streaming PCA and Bayesian learning [2002.08831].

- **Recommender Systems:** DPLR approximations of large parameter matrices in field-weighted factorization machines (FwFMs) drastically reduce inference latency (\(O(F r)\) vs. \(O(F^2)\)), without sacrificing predictive accuracy. Empirical studies show DPLR surpasses parameter-pruning strategies for the same complexity budget, validated in both public benchmark and production ad-serving systems [2408.00801].

- **Kalman Filtering and Gaussian Variational Inference:** The DPLR–projected Riccati flows produce covariance updates with linear storage, yielding state-estimation accuracy superior to pure low-rank approaches [2407.03373].

## 6. Error Analysis, Stability, and Numerical Constraints

Rigorous error and convergence theories underlie DPLR update methods:

- **Spectral and Frobenius Error Bounds:** Backward and forward error in DPLR-updated functions and matrix roots are controlled via residuals of the small projected systems; geometric eigenvalue decay ensures that only a modest rank is needed for accurate approximation [2201.13156, 1707.03045, 2008.11501].

- **Stability Analyses:** All Givens-based transformations in DPLR Hessenberg reduction are stable, and small non-orthogonal corrections apply only to \( k \times k \) subproblems. Empirical studies show that backward error is typically \( O(n \varepsilon_\text{mach}) \) and can be controlled by periodic re-orthogonalization [1501.07812]. In streaming or downdate scenarios, positivity and definiteness are maintained if the number of removed samples does not result in singularities [2002.08831].

- **Rank Selection and Penalty Tuning:** For statistical estimators, cross-validation and AIC-type penalties guide optimal rank choice. The coordinate-descent convergence is monotonic; in practice multiple initializations and warm-starts mitigate the inherent non-convexity of the rank constraint [1802.06048].

## 7. Limitations and Extensions

DPLR methods rely on the assumption that the dominant structure in the data can be well-approximated by low-rank plus diagonal patterns. Performance degrades if the underlying matrix does not exhibit rapid spectral decay or if true interactions are highly non-separable by field, rank, or variable. In non-symmetric or indefinite cases, additional regularization or constraint mechanisms are required. Ongoing extensions include adaptation to factorization machines with complex interaction graphs, online and parallel numerical linear algebra for time-dependent PDEs, and deeper integration with control, system identification, and probabilistic inference.

---

References:
- High-dimensional covariance matrix estimation using a low-rank and diagonal decomposition [1802.06048]
- Quasiseparable Hessenberg reduction of real diagonal plus low rank matrices and applications [1501.07812]
- Low-rank plus diagonal approximations for Riccati-like matrix differential equations [2407.03373]
- Efficiently updating a covariance matrix and its LDL decomposition [2002.08831]
- Low Rank Field-Weighted Factorization Machines for Low Latency Item Recommendation [2408.00801]
- Improved ParaDiag via low-rank updates and interpolation [2204.03073]
- Low-rank updates of matrix functions [1707.03045]
- Low-Rank Updates of Matrix Square Roots [2201.13156]
- Low-rank updates of matrix functions II: Rational Krylov methods [2008.11501]

Source: https://www.emergentmind.com/topics/diagonal-plus-low-rank-dplr-updates