Sherman-Morrison-Woodbury Formula

• The Sherman-Morrison-Woodbury formula is a matrix inversion identity that efficiently computes updates to an inverse when a low-rank modification is applied to a matrix.
• It reduces computational complexity from O(n^3) to O(n^2r + r^3), making it indispensable for fast updates in numerical linear algebra, optimization, and statistical covariance calculations.
• The formula extends to singular and rectangular matrices using Moore-Penrose pseudoinverses and operator-theoretic approaches, broadening its applications in advanced numerical methods.

The Sherman-Morrison-Woodbury (SMW) formula is a matrix inversion identity that provides an explicit, computationally efficient representation for the inverse of a matrix subject to a low-rank update. It is fundamental in linear algebra, numerical linear algebra, statistics, and optimization, due to its utility in handling modifications (often of low rank) to large, structured, or previously factored matrices. Beyond the classical invertible case, SMW admits generalizations to singular matrices, Moore-Penrose pseudoinverses, operator-theoretic contexts, and even third-order tensors under multilinear algebraic products.

1. Classical Formulation and Variants

For a nonsingular $n \times n$ matrix $A$ and matrices $U, V \in \mathbb{C}^{n \times r}$, the Sherman-Morrison-Woodbury formula states:

$$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$

provided $I_r + V^* A^{-1} U$ is invertible. The formula extends immediately to the case $A + U C V$ with $C \in \mathbb{C}^{r \times r}$ invertible:

$$(A + U C V)^{-1} = A^{-1} - A^{-1} U (C^{-1} + V A^{-1} U)^{-1} V A^{-1}$$

The formula is a direct result of the Woodbury matrix identity and can be derived via Schur complements, block matrix inversion, or direct verification through substitution (Arias et al., 2014, Ma et al., 6 Apr 2025, Güttel et al., 2024).

In the rectangular pseudoinverse case, for $A \in \mathbb{R}^{m \times n}$ with full column rank and $U \in \mathbb{R}^{m \times r}$, $A$0, the SMW update for the Moore-Penrose pseudoinverse appears as

$A$1

when $A$2 is invertible (Güttel et al., 2024, Xu, 2016). Generalizations allow for arbitrary projectors and rank augmenting terms (Riedel, 2018).

2. Algebraic and Analytical Foundations

The SMW formula hinges on invertibility conditions for $A$3 and the "capacitance" matrix $A$4. If $A$5 is singular or rectangular, the Moore-Penrose pseudoinverse $A$6 and its generalization via projectors play a central role. The generalized SMW for the pseudoinverse, given suitable range inclusions and closed range of $A$7 and $A$8, reads (Xu, 2016):

$A$9

where $U, V \in \mathbb{C}^{n \times r}$0, $U, V \in \mathbb{C}^{n \times r}$1 are idempotent projectors related to $U, V \in \mathbb{C}^{n \times r}$2 (Xu, 2016). In the fully invertible case these projectors vanish, recovering the classical formula.

Extensions to infinite-dimensional Hilbert space operators and block operator matrices rely on range additivity and compatibility criteria. The Fill–Fishkind formula provides a pseudoinverse update under rank additivity:

$U, V \in \mathbb{C}^{n \times r}$3

with $U, V \in \mathbb{C}^{n \times r}$4, $U, V \in \mathbb{C}^{n \times r}$5 projection operators determined by decompositions of the space according to the kernels and ranges of $U, V \in \mathbb{C}^{n \times r}$6, $U, V \in \mathbb{C}^{n \times r}$7 (Arias et al., 2014).

3. Algorithmic and Computational Implications

SMW enables substantial reductions in computational cost for linear algebra problems with repeated low-rank updates. When $U, V \in \mathbb{C}^{n \times r}$8, the evaluation of $U, V \in \mathbb{C}^{n \times r}$9 via SMW requires $$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$0 arithmetic, as opposed to $$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$1 for direct inversion.

This forms the basis of efficient solvers in statistics (e.g., updating covariance matrices), optimization (e.g., Newton systems with low-rank Hessian changes), numerical PDEs (e.g., domain decomposition or boundary updates), and online machine learning. Notable applications include:

• Fast updates for least squares solutions under low-rank data modifications. The WoodburyLS approach updates the Moore-Penrose pseudoinverse of a data matrix in $$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$2 operations, accelerating iterative solvers and batched optimization routines (Güttel et al., 2024).
• Preconditioning in domain decomposition methods for sparse symmetric systems: low-rank corrections to an approximate inverse via SMW lead to efficient, robust parallel preconditioners (DDLR-1 and DDLR-2) (Li et al., 2015).
• Structured solvers for cyclic banded or heptadiagonal systems, enabling parallelization and constant-time updates of edge couplings (Karawia, 2010).
• Iterative schemes for nearly circulant matrices in finite element and finite difference discretizations, where SMW-based splitting ensures convergence under mild spectral conditions (Mitsotakis, 2023).

A summary comparison of SMW-based updating routines is given below:

Problem Class	SMW Benefit	Asymptotic Complexity
Low-rank update, general square	Fast inversion/update	$$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$3
Rectangular least-squares update	Efficient pseudoinverse	$$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$4
Domain decomposition preconditioner	Robust parallelism	Dominated by low-rank update cost
Cyclic heptadiagonal system	Structured solves	$$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$5 with parallel block solves

4. Stability, Limitations, and Numerical Considerations

While theoretically elegant, the practical application of SMW is sensitive to numerical issues:

• The inversion of $$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$6 can become ill-conditioned, causing both forward and backward errors to amplify quadratically with the condition number of the "capacitance" matrix (Ma et al., 6 Apr 2025). In repeated or iterative use, one must monitor and control the conditioning of this core submatrix.
• For backward stability, the accuracy of the base inverse $$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$7 is crucial; minimizing the error in inverting the small update block is key for forward error.
• Full-column rank conditions on $$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$8 and $$(A + U V^*)^{-1} = A^{-1} - A^{-1} U (I_r + V^* A^{-1} U)^{-1} V^* A^{-1}$$9, as well as invertibility of $I_r + V^* A^{-1} U$0, are strict prerequisites for correctness of the update (Ma et al., 6 Apr 2025).
• In some applications, such as the multiple shooting method for two