KLS Retraction for Low-Rank Matrices

• KLS Retraction is a numerical retraction on the manifold of fixed-rank matrices that maps tangent vectors back onto the manifold with second-order accuracy.
• It employs two QR decompositions to construct the mapping, avoiding expensive full SVDs while maintaining robust performance even with near-zero singular values.
• Derived via Lie–Trotter splitting integrators, it offers an efficient ODE integration tool for dynamical low-rank approximation with reliable numerical stability.

KLS Retraction refers to a numerical retraction operator on the manifold of fixed-rank matrices, designed for manifold-constrained optimization and integration of differential equations, particularly in dynamical low-rank approximation (DLRA) methods. The KLS retraction constructs an explicit mapping from the tangent space at a point on the manifold back to the manifold, facilitating numerical integration within the manifold structure without requiring expensive full singular value decompositions (SVDs). It is motivated by Lie–Trotter splitting integrators and achieves second-order geometric accuracy while maintaining robust numerical stability, even in the presence of small singular values (Séguin et al., 2023).

1. Definition and Construction

Let $M_r \subset \mathbb{R}^{m \times n}$ denote the manifold of matrices of rank exactly $r$. For any $X = U \Sigma V^T \in M_r$ (with $U \in \mathbb{R}^{m \times r}$, $V \in \mathbb{R}^{n \times r}$ orthonormal, $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$) and any tangent vector $Z \in T_X M_r$ (decomposed as $Z = U M V^T + U_\perp V^T + U V_\perp^T$ with $U^T U_\perp = 0$, $V^T V_\perp = 0$), the KLS retraction $r$0 is defined in three steps:

• K-step: Compute a thin QR decomposition

$r$1

and set $r$2.

• L-step: Compute a thin QR decomposition

$r$3

and set $r$4.

• S-step: Form the new middle factor

$r$5

The updated point is then $r$6. Optionally, $r$7 may be re-diagonalized via SVD and the singular vectors absorbed into $r$8 and $r$9 to enforce the canonical form.

2. Derivation via Splitting Integrators

The KLS retraction arises from Lie–Trotter splitting of the vector field associated with projected evolution equations on $X = U \Sigma V^T \in M_r$0. Given a projected ODE of the form $X = U \Sigma V^T \in M_r$1, where $X = U \Sigma V^T \in M_r$2 is the orthogonal projection operator to $X = U \Sigma V^T \in M_r$3, the right-hand side can be split into three components $X = U \Sigma V^T \in M_r$4 corresponding to distinct geometric flows:

• $X = U \Sigma V^T \in M_r$5
• $X = U \Sigma V^T \in M_r$6
• $X = U \Sigma V^T \in M_r$7

Applying forward Euler to each sub-flow sequentially with step size $X = U \Sigma V^T \in M_r$8 yields a combined update that coincides with the KLS retraction:

$X = U \Sigma V^T \in M_r$9

where $U \in \mathbb{R}^{m \times r}$0. This mirrors the "unconventional" integrator for DLRA.

3. Theoretical Properties

The KLS retraction satisfies the standard retraction axioms:

• $U \in \mathbb{R}^{m \times r}$1
• $U \in \mathbb{R}^{m \times r}$2

KLS is a second-order retraction; for the curve $U \in \mathbb{R}^{m \times r}$3, one has $U \in \mathbb{R}^{m \times r}$4, $U \in \mathbb{R}^{m \times r}$5, and $U \in \mathbb{R}^{m \times r}$6. In Riemannian terms,

$U \in \mathbb{R}^{m \times r}$7

Thus, it approximates the Riemannian exponential map to order three, matching the second-order accuracy of the best-known retractions.

4. Algorithmic Implementation and Complexity

The computational steps for $U \in \mathbb{R}^{m \times r}$8 are as follows:

1. QR Decomposition 1: $U \in \mathbb{R}^{m \times r}$9
2. QR Decomposition 2: $V \in \mathbb{R}^{n \times r}$0
3. Small Product: $V \in \mathbb{R}^{n \times r}$1
4. Optional SVD: $V \in \mathbb{R}^{n \times r}$2, update $V \in \mathbb{R}^{n \times r}$3, $V \in \mathbb{R}^{n \times r}$4, $V \in \mathbb{R}^{n \times r}$5
5. Final Composition: $V \in \mathbb{R}^{n \times r}$6

The total cost per retraction is $V \in \mathbb{R}^{n \times r}$7, dominated by two QR decompositions of $V \in \mathbb{R}^{n \times r}$8 and $V \in \mathbb{R}^{n \times r}$9 matrices and small $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$0 matrix operations. KLS avoids inverting potentially ill-conditioned diagonal matrices and remains stable even as $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$1.

5. Comparison with Other Low-Rank Retractions

Retraction	Order	Cost per Step	Robustness to Small $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$2
SVD (metric proj.)	2	Full truncated SVD, $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$3	Robust
Polar	2	Full polar decompositions	Robust
Orthographic	2	Two QRs $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$4 $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$5 inversion	Robust
KLS	2	Two QRs $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$6 small $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$7 ops	Robust
Exponential map	$$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$8	Infinite series or costly implicit	Robust

KLS requires no large SVDs nor $$\Sigma = \operatorname{diag}(\sigma_1,\ldots,\sigma_r) > 0$$9 inversions. It is as accurate (second-order) as SVD, polar, and orthographic retractions, and can be parallelized across the K and L steps. It lacks a closed-form inverse, unlike orthographic retraction, but inversion is not required for most DLRA and integration tasks.

6. Numerical Performance and Stability

In numerical experiments involving DLRA (e.g., Lyapunov equations), KLS retraction-based integrators (KLS, KSL, PRK1) exhibit near-identical performance in both accuracy and computational cost in the first-order regime. For instance, average CPU times per step for $Z \in T_X M_r$0, $Z \in T_X M_r$1 were 5.27 ms (PRK1), 5.41 ms (KSL), and 5.88 ms (KLS). All schemes show the expected first-order error decay with step size.

In scenarios where the smallest singular value $Z \in T_X M_r$2 approaches zero, error-versus-step-size curves remain stable across different ranks, demonstrating that KLS is robust against near-singular situations. This property arises from the QR-based construction, which does not require inversion of $Z \in T_X M_r$3.

7. Context and Application

KLS retraction provides a computationally efficient, explicitly second-order, and numerically stable tool for integrating ODEs on the fixed-rank matrix manifold. The explicit structure is suitable for large-scale DLRA computations, where the cost of full SVDs is prohibitive and robust handling of small singular values is essential. Its derivation from splitting integrators also provides a conceptual link between numerical analysis and geometric integration techniques for constrained dynamical systems (Séguin et al., 2023).