---
title: Low-Rank Lyapunov ADI Methods
url: https://www.emergentmind.com/topics/low-rank-lyapunov-adi-lr-adi
type: topic
---

# Low-Rank Lyapunov ADI Methods

Searching arXiv for recent and foundational papers on LR-ADI, shift selection, inexact solves, projection interpretations, and recent extensions.
Low-Rank Lyapunov ADI (LR-ADI) is the low-rank realization of the alternating direction implicit method for large-scale continuous-time Lyapunov equations, especially when the coefficient matrices are sparse and the constant term has low rank. Instead of forming the dense solution matrix explicitly, LR-ADI computes a factorized approximation, typically \(X \approx ZZ^*\) in the positive semidefinite case or, more generally, an \(LDL^T\)-type representation, so that storage and arithmetic scale with the numerical rank rather than with \(n^2\). Modern analysis places LR-ADI simultaneously in the traditions of stationary matrix iterations, rational Krylov projection, and interpolation-based model reduction, rather than treating it as only a fixed-point recurrence [1309.3985][1201.4779].

## 1. Problem class and algebraic setting

The standard Lyapunov equation treated by LR-ADI is
\[
AX+XA^T=-BB^T,
\]
with \(A\in\mathbb R^{n\times n}\) stable and \(B\in\mathbb R^{n\times r}\), \(r\ll n\). A generalized form,
\[
A P E^T + E P A^T + B B^T = 0,
\]
is equally central in descriptor-system settings, under the assumptions that \(E\) is nonsingular and the pencil \((A,E)\) is asymptotically stable [1309.3985][1809.06903]. In both cases the exact solution is typically dense, while the right-hand side is low rank and the solution often has rapidly decaying singular values, which motivates approximations of the form
\[
X \approx ZZ^T,\qquad Z\in\mathbb R^{n\times s},\quad s\ll n.
\]

This factor viewpoint is the defining feature of LR-ADI. The method never needs to assemble the full dense solution, and its dominant operations are sparse shifted linear solves with a small number of right-hand sides. In positive semidefinite settings the factorization is usually Cholesky-like, but several extensions use symmetric-indefinite forms \(X\approx ZYZ^T\) or \(X\approx LDL^T\) when the initial value is nonzero, the right-hand side is indefinite, or the formulation is generalized [2406.13477][2512.04983].

A source of notational confusion is sign convention. One common formulation uses shifts \(\sigma_i\in\mathbb C_+\) and solves with \(A-\sigma_iE\); another uses \(\alpha_j\in\mathbb C_-\) and solves with \(A+\alpha_jM\). These are equivalent parameterizations of the same basic mechanism, and the literature moves freely between them [1309.3985][1809.06903].

## 2. Core LR-ADI recurrence and residual factors

In generalized form, a standard LR-ADI block recurrence is
\[
Z_1=\sqrt{2\,\operatorname{Re}(\sigma_1)}\,(A-\sigma_1E)^{-1}B,
\]
\[
Z_i=\sqrt{\frac{\operatorname{Re}(\sigma_i)}{\operatorname{Re}(\sigma_{i-1})}}
\left(I+(\sigma_i+\bar\sigma_{i-1})(A-\sigma_iE)^{-1}E\right)Z_{i-1},
\qquad i=2,\dots,k,
\]
after which the approximation is \(\widehat P=ZZ^*\) with \(Z=[Z_1,\dots,Z_k]\). In the alternative residual-factor form one solves
\[
(A+\alpha_jM)v_j=w_{j-1},\qquad
w_j=w_{j-1}-2\operatorname{Re}(\alpha_j)Mv_j,
\]
and appends \(\sqrt{-2\operatorname{Re}(\alpha_j)}\,v_j\) to the low-rank factor [1309.3985][1809.06903].

The residual factorization is one of LR-ADI’s most important computational properties. For the generalized Lyapunov residual
\[
R:=A\widehat P E^T+E\widehat P A^T+BB^T,
\]
the exact LR-ADI iterate satisfies
\[
R=B_\perp B_\perp^T,\qquad
B_\perp = B + E Z L_{\mathrm{ADI}}^T,
\]
so \(\operatorname{rank}(R)\le m\), where \(m\) is the number of columns of \(B\). Hence
\[
\|R\|_2=\|B_\perp^T B_\perp\|_2,
\]
which reduces residual-norm evaluation to an \(m\times m\) problem. In residual-factor notation this is the familiar identity \(\mathcal R_j=w_jw_j^*\), again showing that the residual rank is bounded by the right-hand-side rank [1309.3985][1809.06903].

This residual structure is often mistaken for a secondary implementation detail. It is not. It is the mechanism that makes LR-ADI viable at scale: residual monitoring, stopping criteria, and many adaptive strategies rely on the fact that both the approximation and the residual remain available in low-rank factored form.

## 3. Rational Krylov, projection, and \(\mathcal H_2\) pseudo-optimality

A central structural result is that LR-ADI iterates live in rational Krylov subspaces. For Sylvester and Lyapunov equations with low-rank right-hand side, the ADI factors satisfy
\[
\operatorname{span}(Z)=\mathcal K(A,B,\sigma)
=\operatorname{span}\{(A-\sigma_1E)^{-1}B,\dots,(A-\sigma_kE)^{-1}B\},
\]
so the computed basis is not ad hoc: it is a rational block Krylov basis [1309.3985].

This observation leads to a precise projection interpretation. For arbitrary shift sets closed under conjugation, there always exists an oblique test basis \(W\) such that the LR-ADI approximation coincides with the solution of a projected Lyapunov equation,
\[
\widehat P = V P_q V^T,
\]
with \(V\) spanning the same rational block Krylov space as the ADI basis. Thus LR-ADI is always an implicitly oblique rational Krylov projection method, not only for special shifts. The associated reduced explicit system is unique, and the reduced model attached to the ADI iteration is \(\mathcal H_2\) pseudo-optimal within the class of systems having the same reduced pole structure [1309.3985].

A sharper statement holds for special shifts. For rank-one Sylvester and Lyapunov equations, if the shifts are chosen so that the reduced poles are the mirror images of the interpolation points,
\[
\lambda(Q_r^*AQ_r)=-\{\sigma_1,\dots,\sigma_r\},
\]
then the ADI approximation and the rational Krylov projection approximation coincide exactly. In the Lyapunov case, the corresponding residual satisfies the orthogonality condition
\[
Q_r^*R=0.
\]
These shifts are called pseudo-\(\mathcal H_2\)-optimal because they enforce the interpolation condition associated with \(\mathcal H_2\)-optimal model reduction, but not the derivative-matching condition required for full \(\mathcal H_2\) optimality [1201.4779].

One common misconception is that LR-ADI and projection methods are merely heuristically related. The projection interpretation is exact: always oblique in the general case, and orthogonal under the mirror-pole condition above [1309.3985][1201.4779].

## 4. Shift parameters and convergence behavior

Shift selection governs LR-ADI far more strongly than the outer recurrence itself. The same update formulas can converge rapidly or stagnate depending on the poles. Pseudo-\(\mathcal H_2\)-optimal shifts are one theoretically distinguished class: they make ADI coincide with rational Krylov projection and, in the Lyapunov case, enforce residual orthogonality. Numerically, they also produce nearly best rank-\(r\) approximations in spectral norm on several benchmark problems. In the EADY example, for instance, the best rank-20 benchmark was \(3.38\times 10^{-7}\), while the pseudo-\(\mathcal H_2\) approximation gave \(4.99\times 10^{-7}\); at rank 30 the values were \(7.63\times 10^{-9}\) and \(8.46\times 10^{-9}\), respectively [1201.4779].

A different line of work selects each shift by minimizing the next residual norm. If \(W_j\) is the current residual factor, the ideal next shift minimizes
\[
\psi_j(\alpha)=\|\mathcal C(A,\alpha)W_j\|^2,
\]
but exact evaluation is too expensive because it would require repeated large shifted solves inside a nonlinear optimization loop. The practical solution is to optimize compressed objective functions built from small subspaces already generated during LR-ADI, such as recent columns of the low-rank factor or small extended Krylov spaces. On nonsymmetric examples these residual-minimizing shifts consistently outperform precomputed heuristics and usually outperform competing dynamic strategies, though at the price of more elaborate shift generation [1811.05500].

The paper on adaptive extended rational Krylov approximation adds a related reduced-model shift rule,
\[
s=q^*Bq,
\]
where \(B=U^*AU\) is the projected matrix and \(q\) is extracted from the last row of the reduced Lyapunov solution. Although this result is derived for a rational Krylov method rather than LR-ADI itself, it reinforces the broader principle that useful shifts can be extracted from the current reduced dynamics rather than from static spectral estimates alone [1410.3335].

Another misconception is that shift selection can be based solely on eigenvalue enclosure. For nonsymmetric problems, the literature explicitly criticizes purely spectral heuristics for ignoring \(BB^T\), eigenvector information, and the current residual state [1811.05500].

## 5. Inexact solves, warm starts, integrated Krylov-ADI, and mixed precision

For very large problems, the shifted systems inside LR-ADI are often solved iteratively rather than by sparse direct factorization. In that setting the outer iteration becomes inexact. The exact residual is no longer the computed surrogate \(w_jw_j^*\); instead, a structured residual gap appears, driven by the accumulated inner linear-system residuals. This leads to a precise inexactness model and to practical relaxation rules for the inner stopping tolerance. The preferred adaptive rule uses the computed residual and an accumulated residual-gap estimate to loosen the inner tolerances as the outer iteration progresses. Across the reported examples, relaxed inner solves reduced inner iteration counts by roughly \(14\%\) to \(42\%\), with the second practical strategy consistently better than the first [1809.06903].

The traditional requirement \(X_0=0\) has also been removed. By factorizing the initial residual of an arbitrary low-rank initial guess \(X_0=Z_0Y_0Z_0^H\) as
\[
\Lyap(X_0)=R_0 T R_0^H,
\]
with fixed, generally indefinite \(T\), the standard low-rank increment and residual recurrences extend to nonzero starts. This matters in outer iterations such as Newton methods for algebraic Riccati equations and Rosenbrock schemes for differential Riccati equations, where warm starts from the previous outer iterate can drastically reduce the number of inner LR-ADI steps. Reported speed-ups were about \(17\%\) for the algebraic Riccati setting and up to \(8\times\) for the differential Riccati setting [2406.13477].

A different acceleration route is to merge LR-ADI with an extended Krylov solver for the shifted systems. Because the right-hand sides \(W_j\) of all ADI inner solves stay in a single growing extended Krylov space \(\mathcal{EK}_m(A,B)\), one can reuse one basis for all shifted systems and perform the LR-ADI iteration in projected coordinates. On a class of standard Lyapunov equations this integrated Krylov-ADI approach achieved up to about \(50\%\) speed-up over direct-solver-based LR-ADI, while retaining compatibility with modern shift strategies [2203.17174].

Mixed precision addresses memory rather than iteration count. A recent study split LR-ADI storage and arithmetic across three precision classes: the accumulated factor \(Z\), the propagated factors \(V_k,R_k\), and the inner factors \(T,Y\). The most successful compromise was to store only \(Z\) in single precision while keeping the shifted solves and residual propagation in double precision, denoted ADI(S,D,D). Lowering the precision of \(V_k,R_k\) and the shifted solves caused much larger deterioration in explicit residuals, while keeping only \(T,Y\) in double precision provided essentially no benefit. A particularly important warning is that the implicit LR-ADI residual can remain excellent even when the explicit residual has stagnated badly in finite precision [2508.00722].

## 6. Indefinite, tangential, and unified extensions

Classical block LR-ADI becomes expensive when the right-hand side has large rank or is indefinite. For equations of the form
\[
A X E^H + E X A^H + B R B^H = 0,
\]
with Hermitian indefinite \(R\), a tangential reformulation replaces block updates by rank-1 updates
\[
X_j=X_{j-1}+d_j v_j v_j^H,\qquad
(A+\alpha_j E)v_j=W_{j-1}t_j,
\]
with tangential directions restricted to eigenvectors of \(R\). This reduces per-step rank growth from the full right-hand-side rank to one column, while preserving the low-rank residual structure. The reported numerical results show especially large savings when the constant term has high rank: in one benchmark the tangential approximation had dimension about \(500\) versus about \(16000\) for block ADI [2512.04983].

Recent work has also unified Lyapunov, Sylvester, and Riccati ADI methods under a single recursive Petrov-Galerkin interpolation framework. In this view, the expensive shifted linear solves are shared, while the difference between Lyapunov, Sylvester, and Riccati variants lies mainly in small-scale pole-placement steps. One concrete claim is that two shifted linear solves per iteration suffice to simultaneously solve six Lyapunov equations, one Sylvester equation, and ten Riccati equations; the same framework also exposes the reduced-order models implicitly constructed by the underlying ADI basis [2512.04676].

A broader conceptual extension comes from quadrature and Runge–Kutta formulations. By recasting Lyapunov and Sylvester equations as ODE-driven quadrature problems and imposing preservation of a low-rank residual invariant, one obtains structure-preserving Runge–Kutta schemes whose one-stage instance is exactly equivalent to standard low-rank ADI after the parameter transformation \(\alpha_j=-\mu_j^{-1}\). This does not change the algorithmic core of LR-ADI, but it clarifies why ADI is naturally compatible with low-rank residual factorizations [1903.05383].

Taken together, these developments suggest that LR-ADI is best understood not as a single recurrence but as a family of low-rank, shift-driven, projection-compatible algorithms whose classical Lyapunov form remains the reference case. Its enduring strengths are sparse shifted solves, exact low-rank residual formulas, and a deep compatibility with rational approximation and reduced-order modeling; its persistent challenges are shift generation, large-scale robustness, and extensions to higher-rank, indefinite, or multiterm settings [1309.3985][2512.04676].

Source: https://www.emergentmind.com/topics/low-rank-lyapunov-adi-lr-adi