Nyström Methods: Foundations & Applications

• Nyström Methods are techniques for low-rank matrix approximation that project a large matrix onto a subset of its columns to create a tractable subproblem.
• They are widely applied in kernel methods, numerical ODE integration, and boundary integral equations to achieve scalable, efficient computations with quantifiable error bounds.
• The approach leverages sampling strategies, including uniform, leverage-score, and volume-based methods, to extend the solution of a small problem to the full matrix accurately.

The Nyström method is a foundational set of techniques for low-rank approximation and efficient computation with large matrices, particularly in kernel methods, integral equation solvers, and numerical ODE integration. Its core principle is to approximate a large matrix or operator by projecting onto the span of a judiciously chosen subset of its columns (or rows), thereby inducing a small, computationally manageable subproblem whose solution is then extended to the full matrix. This methodology yields scalable, explicit approximations with theoretically quantifiable error properties, and impacts broad domains in scientific computing, machine learning, and statistics.

1. Core Principle and Standard Formulation

The classical Nyström method targets the approximation of a symmetric positive semidefinite (SPSD) matrix $K \in \mathbb R^{n \times n}$ (such as a kernel or Gram matrix) by sampling a subset $S \subset \{1, \dots, n\}$ of $m \ll n$ indices. Defining

$$C = K_{:,S} \in \mathbb R^{n \times m}, \quad W = K_{S,S} \in \mathbb R^{m \times m},$$

the Nyström approximation is

$K \approx \widetilde K := C W^+ C^T,$

where $W^+$ denotes the Moore–Penrose pseudoinverse (Dereziński et al., 2020, Musco et al., 2016, Bucci et al., 19 Nov 2025). This approximation projects $K$ onto the column space of $C$ and preserves the action of $K$ on the sampled subspace exactly when $W$ is invertible.

When approximating general matrices, one symmetrizes as necessary to recover self-adjoint structure (Nemtsov et al., 2013). For integral operators (e.g., in boundary integral equations), Nyström's approach discretizes the operator using quadrature at a set of collocation sites, leading to tractable, high-accuracy numerical methods (Dominguez et al., 2015).

2. Theoretical Guarantees and Sampling Schemes

Error bounds for the Nyström method generally relate the approximation quality to the spectrum of the target matrix and the sampling strategy. When columns are selected using uniform random sampling or with more sophisticated leverage-score or volume-based schemes, the expected approximation error can be quantified in spectral, Frobenius, or trace norms.

Worst-case guarantee: For a matrix $S \subset \{1, \dots, n\}$0 (or kernel (