---
title: KLS Retraction for Low-Rank Matrices
url: https://www.emergentmind.com/topics/kls-retraction
type: topic
---

# KLS Retraction for Low-Rank Matrices

KLS Retraction refers to a numerical retraction operator on the manifold of fixed-rank matrices, designed for manifold-constrained optimization and integration of differential equations, particularly in dynamical low-rank approximation (DLRA) methods. The KLS retraction constructs an explicit mapping from the tangent space at a point on the manifold back to the manifold, facilitating numerical integration within the manifold structure without requiring expensive full singular value decompositions (SVDs). It is motivated by Lie–Trotter splitting integrators and achieves second-order geometric accuracy while maintaining robust numerical stability, even in the presence of small singular values [2309.06125].

## 1. Definition and Construction

Let $M_r \subset \mathbb{R}^{m \times n}$ denote the manifold of matrices of rank exactly $r$. For any $X = U \Sigma V^T \in M_r$ (with $U \in \mathbb{R}^{m \times r}$, $V \in \mathbb{R}^{n \times r}$ orthonormal, $\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$) and any tangent vector $Z \in T_X M_r$ (decomposed as $Z = U M V^T + U_\perp V^T + U V_\perp^T$ with $U^T U_\perp = 0$, $V^T V_\perp = 0$), the KLS retraction $R_X^{KLS}: T_X M_r \to M_r$ is defined in three steps:

- **K-step:** Compute a thin QR decomposition
  $$
  Q_U S_U = U(\Sigma + M) + U_\perp
  $$
  and set $U_1 = Q_U$.
- **L-step:** Compute a thin QR decomposition
  $$
  Q_V S_V = V(\Sigma + M)^T + V_\perp
  $$
  and set $V_1 = Q_V$.
- **S-step:** Form the new middle factor
  $$
  \Sigma_1 = U_1^T (X + Z) V_1
  $$

The updated point is then $Y_1 = U_1 \Sigma_1 V_1^T$. Optionally, $\Sigma_1$ may be re-diagonalized via SVD and the singular vectors absorbed into $U_1$ and $V_1$ to enforce the canonical form.

## 2. Derivation via Splitting Integrators

The KLS retraction arises from Lie–Trotter splitting of the vector field associated with projected evolution equations on $M_r$. Given a projected ODE of the form $\dot{Y} = \Pi(Y) F(Y)$, where $\Pi(Y)$ is the orthogonal projection operator to $T_Y M_r$, the right-hand side can be split into three components $f_K, f_L, f_S$ corresponding to distinct geometric flows:

- $f_K(Y) = (I - U U^T) F(Y) V V^T$
- $f_L(Y) = U U^T F(Y)$
- $f_S(Y) = -U U^T F(Y) V V^T$

Applying forward Euler to each sub-flow sequentially with step size $h=1$ yields a combined update that coincides with the KLS retraction:
$$
R_X^{KLS}(Z) = (Id + f_S) \circ (Id + f_L) \circ (Id + f_K)(X)
$$
where $Z = \Pi(X)F(X)$. This mirrors the "unconventional" integrator for DLRA.

## 3. Theoretical Properties

The KLS retraction satisfies the standard retraction axioms:

- $R_X^{KLS}(0) = X$
- $D_Z R_X^{KLS}(0)[Z] = Z$

KLS is a *second-order retraction*; for the curve $\sigma(t) = R_X^{KLS}(t Z)$, one has $\sigma(0) = X$, $\sigma'(0) = Z$, and $\sigma''(0) = 0$. In Riemannian terms,
$$
R_X^{KLS}(t Z) = \operatorname{Exp}_X(t Z) + O(t^3)
$$
Thus, it approximates the Riemannian exponential map to order three, matching the second-order accuracy of the best-known retractions.

## 4. Algorithmic Implementation and Complexity

The computational steps for $R_X^{KLS}(Z)$ are as follows:

1. **QR Decomposition 1:** $(Q_U, S_U) = \operatorname{qr}(U(\Sigma + M) + U_\perp)$
2. **QR Decomposition 2:** $(Q_V, S_V) = \operatorname{qr}(V(\Sigma + M)^T + V_\perp)$
3. **Small Product:** $\Sigma_1 = Q_U^T (X+Z) Q_V$
4. **Optional SVD:** $[\tilde{U}, \tilde{\Sigma}, \tilde{V}] = \operatorname{svd}(\Sigma_1)$, update $U_1 = Q_U \tilde{U}$, $V_1 = Q_V \tilde{V}$, $\Sigma_1 = \tilde{\Sigma}$
5. **Final Composition:** $Y_1 = U_1 \Sigma_1 V_1^T$

The total cost per retraction is $O((m+n)r^2 + r^3)$, dominated by two QR decompositions of $m \times r$ and $n \times r$ matrices and small $r \times r$ matrix operations. KLS avoids inverting potentially ill-conditioned diagonal matrices and remains stable even as $\sigma_r(X) \to 0$.

## 5. Comparison with Other Low-Rank Retractions

| Retraction         | Order | Cost per Step                       | Robustness to Small $\sigma_r(X)$ |
|--------------------|-------|-------------------------------------|-----------------------------------|
| SVD (metric proj.) | 2     | Full truncated SVD, $O(m n r)$      | Robust                            |
| Polar              | 2     | Full polar decompositions           | Robust                            |
| Orthographic       | 2     | Two QRs $+$ $r \times r$ inversion  | Robust                            |
| KLS                | 2     | Two QRs $+$ small $r \times r$ ops  | Robust                            |
| Exponential map    | $\infty$ | Infinite series or costly implicit | Robust                            |

KLS requires no large SVDs nor $r \times r$ inversions. It is as accurate (second-order) as SVD, polar, and orthographic retractions, and can be parallelized across the K and L steps. It lacks a closed-form inverse, unlike orthographic retraction, but inversion is not required for most DLRA and integration tasks.

## 6. Numerical Performance and Stability

In numerical experiments involving DLRA (e.g., Lyapunov equations), KLS retraction-based integrators (KLS, KSL, PRK1) exhibit near-identical performance in both accuracy and computational cost in the first-order regime. For instance, average CPU times per step for $m = n = 100$, $r = 12$ were 5.27 ms (PRK1), 5.41 ms (KSL), and 5.88 ms (KLS). All schemes show the expected first-order error decay with step size.

In scenarios where the smallest singular value $\sigma_r$ approaches zero, error-versus-step-size curves remain stable across different ranks, demonstrating that KLS is robust against near-singular situations. This property arises from the QR-based construction, which does not require inversion of $\Sigma$.

## 7. Context and Application

KLS retraction provides a computationally efficient, explicitly second-order, and numerically stable tool for integrating ODEs on the fixed-rank matrix manifold. The explicit structure is suitable for large-scale DLRA computations, where the cost of full SVDs is prohibitive and robust handling of small singular values is essential. Its derivation from splitting integrators also provides a conceptual link between numerical analysis and geometric integration techniques for constrained dynamical systems [2309.06125].

Source: https://www.emergentmind.com/topics/kls-retraction