---
title: Eckart–Young–Mirsky Theorem
url: https://www.emergentmind.com/topics/eckart-young-mirsky-theorem
type: topic
---

# Eckart–Young–Mirsky Theorem

The Eckart–Young–Mirsky theorem precisely characterizes the best low-rank approximation of a matrix under any unitarily invariant norm, establishing that truncation of the singular value decomposition (SVD) yields the unique optimal solution. This result underpins a wide range of applications in numerical linear algebra, data analysis, optimization, and deep learning. Modern research further extends its scope to tensors, tubal tensor frameworks, autoencoders, and the stability of low-rank approximations under perturbations.

## 1. Classical Statement and Generalization under Unitarily Invariant Norms

Let $A\in\mathbb{R}^{m\times n}$ with SVD $A=U\,\mathrm{diag}(\sigma_1,\dots,\sigma_r,0,\dots,0)\,V^*$, where $\sigma_1\ge\sigma_2\ge\cdots\ge\sigma_r>0$ and $r=\mathrm{rank}(A)$. The truncated SVD of rank $k\le r$ is defined as $A_k = U\,\mathrm{diag}(\sigma_1,\dots,\sigma_k,0,\dots,0)\,V^*$. The original theorem states:
- In the Frobenius norm,
  $$
  \|A - A_k\|_F = \min_{\mathrm{rank}(B)\le k} \|A-B\|_F = \left( \sum_{i=k+1}^r \sigma_i^2 \right)^{1/2}.
  $$
- In the spectral (operator) norm,
  $$
  \|A - A_k\|_2 = \min_{\mathrm{rank}(B)\le k} \|A-B\|_2 = \sigma_{k+1}.
  $$
The minimizer is unique in either norm if and only if $\sigma_k > \sigma_{k+1}$ [1202.3772].

A generalization asserts that for any unitarily invariant norm $\|\cdot\|_{\mathrm{UI}}$,
$$
A_k = \arg\min_{\mathrm{rank}(B)\le k} \|A-B\|_{\mathrm{UI}},
$$
meaning the SVD truncation is optimal across all such norms, including Schatten $p$-norms and Ky Fan $k$-norms [1202.3772]. The proof leverages SVD block decomposition, singular value majorization, and monotonicity of unitarily invariant norms.

## 2. Extensions to Tensors

Efforts to generalize the theorem to tensor settings have revealed deeper geometric and algebraic structures. For order-$p$ (partially symmetric) tensors, best rank-$k$ approximations in the Frobenius norm lack a simple universal SVD truncation analogue. Nonetheless, for a "sufficiently general" tensor $f$ in $T$, all critical rank-at-most-$k$ approximations are confined to a fixed critical subspace $H_f$, which is the span of the complex critical rank-one tensors under suitable dimension and symmetry constraints [1711.06443]. This framework encapsulates geometric features such as secant varieties of Segre–Veronese embeddings and recovers the classical theorem for matrices.

For tubal tensors, as in the t-SVD/tubal algebra setting, an Eckart–Young–Mirsky-type result holds precisely for tubal products induced by block-orthogonal (unitary up to scaling) transforms. Here, optimally truncating the t-SVD yields the best low-rank approximation in the Frobenius norm. Necessary and sufficient conditions on the tubal algebra guarantee this result and unify the classical and tensorial statements [2512.24405].

## 3. Role in Optimization, Closed-Form Solutions, and Subspace Clustering

Because the truncated SVD provides the minimizer for general unitarily invariant norms, many rank-constrained or norm-regularized problems admit closed-form solutions:
- For problems of the form $\min_{\mathrm{rank}(X)\le k} \|A - BXC\|_{\mathrm{UI}}$, under suitable conditions on $B$ and $C$, the solution reduces to the SVD truncation of a projected matrix [1202.3772].
- In subspace clustering, the shape-interaction matrix $Z^\star = X^\dagger X$ remains optimal under any unitarily invariant norm, and in noisy or regularized variants, the solution can be constructed via SVD thresholding [1202.3772].

The table below summarizes classic and generalized statements:

| Setting                | Best Rank-$k$ Approximant     | Norm Optimality     | Uniqueness Condition        |
|------------------------|------------------------------|---------------------|-----------------------------|
| Matrices, SVD          | Truncation $A_k$             | Any unitarily inv.  | $\sigma_k > \sigma_{k+1}$   |
| Partially symmetric tensor | Linear span in $H_f$      | Frobenius (in general) | Sufficiently general $f$ |
| Tubal tensors (t-SVD)  | Truncated t-SVD (if algebra conditions met) | Frobenius              | Algebraic; transform block-unitarity |

## 4. Stability under Perturbations and Spectral Bounds

Perturbation analysis quantifies the stability of SVD truncation and resulting low-rank approximants under noise. The classical theorem provides the bound for any perturbation $E$:
$$
\bigl\|\widetilde A_p - A_p\bigr\| \leq 2 (\sigma_{p+1} + \|E\|).
$$
Recent advances yield tighter, high-probability spectral-norm perturbation bounds when $A$ is symmetric with eigengap $\delta_p = \lambda_p - \lambda_{p+1}$ and $4\|E\| \leq \delta_p$:
$$
\bigl\|(A+E)_p - A_p\bigr\| \leq C\,\|E\|\,\frac{\lambda_p}{\delta_p}
$$
where $C < 7$ [2510.25670]. If the noise $E$ aligns weakly with the leading eigenspace, an additional gain is possible. These results are crucial for analyzing differentially private PCA and spectral algorithms in high-dimensional regimes [2510.25670].

## 5. Applications in Deep Learning and Autoencoders

The theorem underlies model compression in deep neural networks via low-rank decompositions. In such frameworks, each layer's weight tensor is unfolded into a matrix, which is then block-wise compressed by SVD truncation. The optimal error bound for each layer, used by allocation algorithms for global compression targets, is given by the Eckart–Young–Mirsky result [2107.11442]. Similarly, in symmetric autoencoders, the theorem yields explicit layer-wise reconstruction error bounds when the weights are orthonormal, and motivates the EYS (Eckart–Young–Schmidt) initialization: sequentially initializing layers by empirical SVD on latent representations [2506.11641].

## 6. Historical and Contemporary Impact

The Eckart–Young–Mirsky theorem, with roots in the 1930s, has become foundational in matrix approximation, data analysis (PCA), numerical algorithms, and convex optimization. Its extension to tensors is the subject of active research. Modern work in tubal tensor algebra, high-dimensional spectral analysis, and deep learning architectures continues to explore its boundaries and leverage its guarantees for both theory and applications [1202.3772, 1711.06443, 2512.24405, 2510.25670, 2506.11641, 2107.11442].

## 7. Connections to Broader Theories and Open Directions

Connections exist to the geometry of secant varieties, module theory in algebra, operator theory in Hilbert spaces, and spectral functionals in matrix analysis. Recent developments address optimality in more general algebras, the ramifications for data privacy, and non-Euclidean approximation problems. Ongoing research investigates sharper perturbation bounds, closed-form solutions in structured settings, and the existence or uniqueness of optimal approximations for higher-order and structured tensors—highlighting both the power and the subtle limitations of the Eckart–Young–Mirsky paradigm across modern computational mathematics.

Source: https://www.emergentmind.com/topics/eckart-young-mirsky-theorem