Nyström–Schur Preconditioners

• Nyström–Schur preconditioners are techniques that approximate the intractable Schur complement using low-rank methods to enable efficient, scalable solutions for large sparse and structured systems.
• They combine principles from domain decomposition, randomized numerical linear algebra, and spectral theory to improve convergence in PDE solvers, SPD matrix preconditioning, and kernel methods.
• The approach offers theoretical spectral guarantees, reduced iteration counts, and practical performance improvements by balancing setup cost with accelerated convergence.

Nyström–Schur preconditioners constitute a family of algebraic preconditioning techniques in which a low-rank (Nyström-type) approximation of a Schur complement, or a closely related matrix, enables the construction of scalable, robust, and spectrally optimized preconditioners for large sparse, structured, or kernel-based linear systems. This paradigm integrates classical domain decomposition, modern randomized numerical linear algebra, and spectral theory. Across multiple regimes—including domain decomposition for PDE solvers, algebraic two-level methods for sparse symmetric positive definite (SPD) matrices, and kernel methods for large datasets—the Nyström–Schur viewpoint abstracts the essential procedure: approximate an intractable Schur complement by a data-efficient, low-rank operator that is fast to construct and apply, yet preserves the spectral features critical for rapid convergence of Krylov or gradient-type solvers.

1. Schur Complement and Low-Rank Approximation Foundations

The general setting begins with a partitioned matrix

$$A = \begin{pmatrix} B & E \ E^T & C \end{pmatrix}$$

with $B \in \mathbb{R}^{n_I \times n_I}$ (interior), $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$ (interface/boundary), and $E \in \mathbb{R}^{n_I \times n_\Gamma}$, arising either from domain decomposition, graph partitioning, or block reordering. The Schur complement for the interface,

$S = C - E^T B^{-1} E,$

is central to leveraging a two-level solver: inversion or accurate preconditioning of $S$ is key to the overall efficiency of the system solver.

Rather than compute $S$ explicitly—which is infeasible for large problems—Nyström–Schur preconditioning replaces $S$ (or $S^{-1}$) with a rank-$k$ or otherwise data-efficient approximation, constructed either via spectral methods (Lanczos, randomized projections) or explicit matrix sampling, in the spirit of the Nyström method (Li et al., 2015, Daas et al., 2021, Abedsoltan et al., 2023, Dereziński et al., 2024).

2. Construction Algorithms

2.1 SLR Framework (Schur Low-Rank Preconditioning)

The SLR method of Li–Xi–Saad implements a spectral low-rank correction to $B \in \mathbb{R}^{n_I \times n_I}$0 by exploiting the following decomposition for $B \in \mathbb{R}^{n_I \times n_I}$1 SPD: $B \in \mathbb{R}^{n_I \times n_I}$2 where $B \in \mathbb{R}^{n_I \times n_I}$3. The algorithm uses $B \in \mathbb{R}^{n_I \times n_I}$4 Lanczos steps on $B \in \mathbb{R}^{n_I \times n_I}$5 to obtain a low-rank approximation $B \in \mathbb{R}^{n_I \times n_I}$6. The preconditioner is formed as

$B \in \mathbb{R}^{n_I \times n_I}$7

and applied to the interface unknowns in a block-solve-backsubstitute manner. The low-rank correction mirrors the Nyström idea: approximate a symmetric positive semidefinite (SPSD) matrix by projecting onto the dominant subspace of $B \in \mathbb{R}^{n_I \times n_I}$8 (Li et al., 2015).

2.2 Two-Level Nyström–Schur for Sparse SPD Matrices

Building on domain decomposition but focusing on randomized algebraic methods, the two-level approach first samples the action of $B \in \mathbb{R}^{n_I \times n_I}$9 with a Gaussian test matrix $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$0: $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$1 and the Nyström approximation is $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$2. $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$3 is replaced by $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$4 within the block-inverse formula for $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$5. All steps are constructed to avoid explicit formation of $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$6; sampling and solves are managed via efficient block operations and Krylov subspace methods (Daas et al., 2021).

2.3 Multi-Level Sketching: SKINNY Framework

The SKINNY approach generalizes Nyström–Schur preconditioning via three levels of sparse sketching. At the first level, a sparse sketch $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$7 and the corresponding Nyström approximation

$C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$8

yield the preconditioner for $C \in \mathbb{R}^{n_\Gamma \times n_\Gamma}$9 as $E \in \mathbb{R}^{n_I \times n_\Gamma}$0. Inverting $E \in \mathbb{R}^{n_I \times n_\Gamma}$1 is performed via further levels of sketch-preconditioned iterative solvers, replacing large matrix factorizations with efficient inner-outer Krylov iterations (Dereziński et al., 2024).

2.4 Kernel Methods: Nyström–Schur for Spectral Preconditioning

Given a kernel matrix $E \in \mathbb{R}^{n_I \times n_\Gamma}$2, fixing a sample $E \in \mathbb{R}^{n_I \times n_\Gamma}$3 of size $E \in \mathbb{R}^{n_I \times n_\Gamma}$4, the classical Nyström approximation,

$E \in \mathbb{R}^{n_I \times n_\Gamma}$5

with $E \in \mathbb{R}^{n_I \times n_\Gamma}$6, $E \in \mathbb{R}^{n_I \times n_\Gamma}$7, becomes a Schur-complement approximation in the block-partitioned $E \in \mathbb{R}^{n_I \times n_\Gamma}$8. The equivalence to replacing the fully dense Schur complement by a low-rank term $E \in \mathbb{R}^{n_I \times n_\Gamma}$9 underpins theoretical and empirical acceleration in preconditioned gradient methods (Abedsoltan et al., 2023).

3. Theoretical Guarantees and Spectral Properties

The efficacy of Nyström–Schur preconditioners is governed by strong spectral bounds. In the SLR framework, the spectrum of the preconditioned Schur complement is explicitly controlled: if the $S = C - E^T B^{-1} E,$0 leading eigenvalues of $S = C - E^T B^{-1} E,$1 are retained, the effective condition number of the preconditioned system is

$S = C - E^T B^{-1} E,$2

with $S = C - E^T B^{-1} E,$3, and the principal subspace dimensions $S = C - E^T B^{-1} E,$4 can be tuned to ensure any target $S = C - E^T B^{-1} E,$5 (Li et al., 2015).

In randomized Nyström–Schur methods, the expected effective condition number obeys

$S = C - E^T B^{-1} E,$6

with $S = C - E^T B^{-1} E,$7 only polynomially dependent on rank/oversampling and decaying with greater oversampling (Daas et al., 2021).

For kernel preconditioning, a sample size $S = C - E^T B^{-1} E,$8 is sufficient to ensure that the preconditioned operator nearly matches the spectrum of the ideal rank-$S = C - E^T B^{-1} E,$9 spectral preconditioner with high probability. Preconditioned gradient (or CG) iteration counts scale with the post-processed condition number, while total per-iteration cost is $S$0 (Abedsoltan et al., 2023).

The multi-level sketched approach formalizes convergence in terms of an average tail condition number, yielding provable iteration and runtime improvements that match or surpass prior stochastic methods for a range of spectral profiles (Dereziński et al., 2024).

4. Computational Complexity and Implementation

For SLR-type methods, preconditioner assembly involves factorizations of subdomain blocks $S$1, Cholesky/ILU of $S$2, and $S$3 Lanczos steps. Each application involves two $S$4 solves, two triangular solves, four sparse matvecs, and $S$5 flops for the low-rank correction. Setup may be more expensive (factor $S$6–$S$7 vs. incomplete Cholesky/ILU), but per-iteration costs are amortized across faster convergence (Li et al., 2015).

Randomized algorithms based on Nyström sampling with block Krylov methods (e.g., block CG for inner solves) exploit matrix-matrix products for sample collection, thin QR, and dense eigenvalue problems of size $S$8. Storage is $S$9 beyond factors of $S$0. The cost model is dominated by $S$1 (Daas et al., 2021).

SKINNY and other multi-level approaches leverage fast sparse embeddings, with work- and storage-bounds scaling as $S$2 per iteration, where $S$3, and offline $S$4 for dense subproblems. All systems reduce inversion or application to $S$5 or similar size (Dereziński et al., 2024).

Kernel-based Nyström–Schur preconditioning stores only $S$6 and $S$7 blocks and incurs $S$8 computation per iteration; this is almost always subdominant compared to a full matrix-vector multiplication for unpreconditioned methods (Abedsoltan et al., 2023).

5. Practical Applications and Numerical Performance

Nyström–Schur preconditioning has demonstrated robust and scalable performance on a diverse class of matrices:

• Finite-difference/element discretizations for 2D/3D elliptic PDEs, with SLR and two-level approaches delivering up to $S$9–$S$0 reductions in iteration time and factor $S$1–$S$2 improvements in iteration counts compared to ILU or RAS (Li et al., 2015).
• Indefinite or shifted SPD problems, where SLR remains robust and classical preconditioners fail or stagnate.
• Kernel ridge regression and large-scale kernel classification, where using $S$3 Nyström samples recovers nearly all spectral acceleration of ideal preconditioners with only mild extra storage and substantial speed-up, both in theory and reflected in empirical regimes such as MNIST with $S$4 (Abedsoltan et al., 2023).
• Matrix norm (Schatten–p) approximation, with sketched Nyström–Schur preconditioning improving state-of-the-art running times for numerous norm estimation tasks (Dereziński et al., 2024).

Numerical experiments confirm the advantage of block and randomized inner solves (e.g., block CG) and the stability of relaxed accuracy parameters for the approximate solvers involved (Daas et al., 2021).

6. Comparative Analysis with Classical and Other Approaches

Relative to classical two-level Schur complement preconditioners—which target small eigenpairs of generalized problems $S$5 through Lanczos on $S$6—the Nyström–Schur approach offers concrete computational benefits:

• The spectra of $S$7 (used in Nyström methods) have leading eigenvalues that are well-separated, making subspace extraction by randomized sampling far more efficient than for near-degenerate classical settings (Daas et al., 2021).
• SKINNY and related methods avoid explicit orthogonalization and large dense solves for $S$8, replacing them with iterative sketch-based inversion, improving scalability and practical implementation (Dereziński et al., 2024).
• Nyström preconditioning in kernel machines provides a principled trade-off between storage/setup and per-iteration complexity that dramatically accelerates convergence over unpreconditioned or naïvely preconditioned methods for large datasets (Abedsoltan et al., 2023).

7. Extensions, Limitations, and Future Directions

Recent developments incorporate multi-level sketching, block randomized solvers, and adaptive sampling schemes to further improve both theoretical and empirical performance. Nyström–Schur methods have been extended to regularized problems $S$9, matrix norm estimation, and beyond. The selection of rank parameter $S^{-1}$0 (or sample size $S^{-1}$1), oversampling, and inner tolerance are tunable knobs that can be set based on problem spectra to guarantee any target condition number or computational resource constraint.

Classical limitations—such as failure on highly indefinite or near-singular problems—are mitigated in the Nyström–Schur paradigm by flexible low-rank adaptation and robust spectrum control. A plausible implication is that further advances in practical random projection algorithms and sparse factorizations will push the scalability and applicability of these preconditioners to even larger and more complex linear systems.

---

References:

• (Li et al., 2015) Schur Complement based domain decomposition preconditioners with Low-rank corrections
• (Daas et al., 2021) Two-level Nyström–Schur preconditioner for sparse symmetric positive definite matrices
• (Dereziński et al., 2024) Faster Linear Systems and Matrix Norm Approximation via Multi-level Sketched Preconditioning
• (Abedsoltan et al., 2023) On the Nyström Approximation for Preconditioning in Kernel Machines