---
title: Greedy Eigen-Based Selection (GES)
url: https://www.emergentmind.com/topics/greedy-eigen-based-selection-ges
type: topic
---

# Greedy Eigen-Based Selection (GES)

Greedy Eigen-Based Selection (GES) denotes a class of greedy procedures that iteratively choose, retain, or discard coordinates, columns, atoms, or parameters using a score derived from explained variance, projection error, residual magnitude, mutual information, or a spectral approximation criterion. The supplied literature suggests that the label does not identify a single canonical algorithm. Instead, it appears in several distinct settings: forward regression for subset selection [1102.3975], greedy generalized column subset selection [1312.6820], localized eigenpair computation from principal submatrices [1911.10041], RF-chain pruning in MIMO integrated sensing and communications [2507.09960], and recursion-parameter choice for One-Way Navier-Stokes approximations [2506.02320]. Across these settings, the common structure is iterative greedy search guided by a reduced spectral surrogate.

## 1. Canonical pattern and scope

In the supplied papers, GES follows a recurring template: maintain a current active set, evaluate a per-candidate score, and greedily update the active set so as to improve an objective or reduce a residual. The objective varies by domain, but the update is always local and myopic rather than combinatorial or exhaustive.

| Setting | Objective | Greedy action |
|---|---|---|
| Subset selection | Maximize $f(S)=b_S^T\Sigma_S^{-1}b_S$ | Add the variable with largest marginal gain |
| Generalized CSS | Minimize $\|B-P_SB\|_F^2$ | Add the column with largest error reduction |
| Sparse eigenvalue computation | Reduce eigenpair residual of a principal submatrix | Add indices with largest residual or perturbation score |
| MIMO ISAC RF-chain selection | Maximize weighted communication/sensing MI | Remove the chain with smallest contribution score |
| OWNS recursion design | Minimize projector-approximation objective | Append the worst downstream and upstream eigenvalues |

This breadth has two immediate consequences. First, “eigen-based” does not always mean repeated full eigendecompositions; in several formulations the decisive quantity is an inverse-update, a residual, or a projection formula. Second, approximation guarantees are not uniform across the literature: some variants admit explicit bounds, while others remain analytically open.

## 2. Forward-regression GES for subset selection

The most explicit early formulation appears in the subset-selection problem of Das and Kempe, where one is given zero-mean, unit-variance variables $V=\{X_1,\ldots,X_n\}$ with covariance matrix $\Sigma\in\mathbb{R}^{n\times n}$ and a target variable $Z$ with covariance vector $b\in\mathbb{R}^n$. For any $S\subseteq V$, the explained variance is
$$
R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,
$$
and the subset-selection problem is to find $S\subseteq V$, $|S|\le k$, maximizing
$$
f(S):=R^2(Z;S)=b_S^T\Sigma_S^{-1}b_S.
$$
GES is “essentially ‘Forward Regression’,” selecting at each step the variable with the largest increase $\Delta(j\mid S)=f(S\cup\{X_j\})-f(S)$ [1102.3975].

The analysis centers on the submodularity ratio
$$
\gamma_{U,k}:=\min_{L\subseteq U,\;S\subseteq V\setminus L,\;|S|\le k}
\frac{\sum_{x\in S}(f(L\cup\{x\})-f(L))}{f(L\cup S)-f(L)},
$$
together with the smallest $k$-sparse eigenvalue
$$
\lambda_{\min}(k):=\min_{v\ne 0,\;\|v\|_0\le k}\frac{v^T\Sigma v}{v^Tv}.
$$
If $S_g$ is the greedy $k$-set and $S^*$ is an optimal $k$-set, then the recurrence
$$
f(S_{i+1})-f(S_i)\ge (\gamma/k)\cdot(f^*-f(S_i))
$$
implies
$$
f(S_k)\ge \bigl(1-(1-\gamma/k)^k\bigr)f^*\ge (1-e^{-\gamma})f^*.
$$
Lemma 3.2 further shows $\gamma\ge \lambda_{\min}(\Sigma,2k)$, yielding
$$
f(S_g)\ge \bigl(1-e^{-\lambda_{\min}(\Sigma,2k)}\bigr)f(S_{\mathrm{opt}}).
$$
A central significance of this result is that the guarantee is controlled by approximate submodularity rather than only by sparse-eigenvalue or RIP-style conditions.

The computational structure is also explicit. With a maintained Cholesky factorization or inverse update of $\Sigma_S$, each candidate marginal gain can be computed in $O(k^2)$, so each iteration costs $O(nk^2)$ and the total runtime is $O(nk^3)$. Experiments on Boston housing $(n=15)$, World Bank indicators $(n=29)$, and synthetic Gaussian data $(n=29)$, with $k$ up to $8$, show that GES achieves near-optimal $R^2$ on all sets, within $1$–$2\%$ of the exact optimum; OMP is slightly worse, while oblivious selection and $L_1$ relaxation are notably weaker. The paper also reports that $\lambda_{\min}(\Sigma,2k)$ and condition-number/RIP bounds are often near zero on nearly singular data, whereas the observed submodularity ratio $\gamma$ is substantially larger, often $0.2$–$1$, and correlates much better with the actual $R^2$ gaps [1102.3975].

## 3. Generalized column subset selection and sparse approximation

A second important usage concerns generalized column subset selection. Here $A\in\mathbb{R}^{m\times n}$ is the source matrix, $B\in\mathbb{R}^{m\times r}$ is the target matrix, and the goal is to choose $S\subset\{1,\ldots,n\}$, $|S|=\ell$, minimizing
$$
\|B-P_SB\|_F^2,
\qquad
P_S=A_{:S}(A_{:S}^TA_{:S})^{-1}A_{:S}^T.
$$
Equivalently,
$$
S^*=\underset{|S|=\ell}{\arg\min}\;
\big\|B-A_{:S}(A_{:S}^TA_{:S})^{-1}A_{:S}^TB\big\|_F^2.
$$
The sectioned summary notes that the paper itself “never uses the name GES explicitly,” but the method is a greedy, eigen-inspired column-selection routine for the generalized CSS problem [1312.6820].

At iteration $t$, the residuals are
$$
E=A-P_SA,\qquad F=B-P_SB,
$$
with current error $\mathcal{F}(S)=\|F\|_F^2$. Adding column $i$ reduces the error by
$$
\Delta_i=\|P_{\{i\}}F\|_F^2
=\frac{\|F^TE_{:i}\|_2^2}{E_{:i}^TE_{:i}}.
$$
If $H=F^TE$ and $G=E^TE$, then this becomes
$$
\Delta_i=\frac{H_{:i}^TH_{:i}}{G_{ii}}.
$$
The algorithm therefore selects
$$
p=\arg\max_i f_i/g_i,
$$
where $f_i=\|H_{:i}\|^2$ and $g_i=G_{ii}$, and updates these quantities by rank-one recursions derived in Theorem 3. The resulting implementation precomputes $A^TB$ once, then uses incremental updates rather than recomputing full residual statistics.

Its theory is structurally different from the subset-selection case. The paper proves a projection recursion, $P_S=P_P+P_R(E)$ when $S=P\cup R$, and an error recursion,
$$
\mathcal{F}(S)=\mathcal{F}(P)-\|P_R(E)F\|_F^2.
$$
However, it “does not prove a constant-factor or relative-error bound vs. the optimal $\ell$-subset,” and it remains open whether the greedy rule yields a $(1-1/e)$ bound or similar. Runtime is $O(\max(mn\ell,mnr))$, or $O(sn)$ per step when $A$ is sparse with $s$ nonzeros per column. Several specializations are exact: setting $B:=A$ gives ordinary CSS; taking $B=U_k\Sigma_k$ recovers the greedy SVD-based CSS method of Çivril and Magdon-Ismail; and if $B$ is a single vector $y\in\mathbb{R}^m$, the procedure becomes Orthogonal Least Squares [1312.6820].

## 4. Localized eigenpair extraction and high-dimensional eigenvalue problems

In large sparse symmetric eigenproblems, GES denotes a method that exploits localization of the target eigenvector. Given $H\in\mathbb{R}^{n\times n}$ symmetric, the algorithm maintains an “important” index set $\mathcal{I}_t$, forms the principal submatrix $H_t=H_{\mathcal{I}_t\mathcal{I}_t}$, computes its smallest eigenpair
$$
H_tx_t=\lambda_t x_t,\qquad \|x_t\|=1,
$$
embeds $x_t$ into the full space as $\hat x_t=\begin{bmatrix}x_t\\0\end{bmatrix}$, and evaluates the residual
$$
r_t=H\hat x_t-\lambda_t\hat x_t
=\begin{bmatrix}0\\B_t^Tx_t\end{bmatrix}.
$$
Candidates in the complement $\mathcal{J}_t$ are then scored either by the residual-based quantity
$$
s^{(\mathrm{res})}_{t,j}=|(B_t^Tx_t)_j|
$$
or by the perturbation-based quantity
$$
s^{(\mathrm{pert})}_{t,j}
=\left|\frac{(B_t^Tx_t)_j}{\lambda_t-[C_t]_{j,j}}\right|,
$$
and the top $k$ indices are appended to $\mathcal{I}_t$ [1911.10041].

The spectral interpretation is precise. By the interlacing theorem for principal submatrices, the reduced eigenvalue satisfies $\lambda_{\mathcal{I}}\ge \lambda=\min\{\mathrm{spec}(H)\}$. Error control is residual-driven:
$$
|\lambda_t-\lambda|\le \|r_t\|,
\qquad
\sin\theta\le \frac{\|r_t\|}{\delta}
$$
when the target eigenvalue is simple and $\delta$ is the spectral gap. The approximation can then serve as an initializer for LOBPCG or Lanczos. Complexity depends on the current active-set size $m=|\mathcal{I}|$: solving the dense reduced problem costs $O(m^3)$, computing $B_t^Tx_t$ costs $O(\mathrm{nnz}(B_t))$, and selecting the top $k$ scores costs $O((n-m)\log k)$. The reported examples include nuclear configuration-interaction, many-body-localization spin chains, and California road-network analysis, with speedups ranging from $1.9\times$–$2.5\times$ against direct LOBPCG, to $7\times$ faster than full-matrix `eigs`, to an overall $420\times$ speedup over full-matrix Lanczos when localization is especially sharp [1911.10041].

A related but more variational line of work concerns high-dimensional eigenvalue problems in tensor-product spaces. Cancès, Ehrlacher, and Lelievre introduce the Pure Rayleigh Greedy Algorithm (PRaGA) and Pure Residual Greedy Algorithm (PReGA), together with orthogonalized versions ORaGA, OReGA, and OEGA. In a Hilbert-space setting with bilinear form $a(\cdot,\cdot)$ and Rayleigh quotient
$$
R(v)=\frac{a(v,v)}{\|v\|_{L^2}^2},
$$
PRaGA chooses
$$
z_n\in\arg\min_{z\in\Sigma} R(u_{n-1}+z),
$$
while PReGA chooses
$$
z_n\in\arg\min_{z\in\Sigma}
\Bigl\{\tfrac12\|u_{n-1}+z\|_a^2-(\lambda_{n-1}+\nu)\langle u_{n-1},z\rangle\Bigr\},
$$
normalizing after each update. Under compact-embedding and coercivity assumptions, these methods are well posed, satisfy $\lambda_n\downarrow \lambda$, and converge strongly in $V$ to an $H$-normalized eigenvector. In finite dimension, a Łojasiewicz argument yields either geometric decay $\|u_n-w\|_a\le C\sigma^n$ when $\theta=\tfrac12$ or algebraic decay $\|u_n-w\|_a\le Cn^{-\theta/(1-2\theta)}$ when $\theta\in(0,\tfrac12)$. Numerical tests on a toy SPD-matrix problem and on buckling modes of a microstructured plate exhibit roughly exponential decay of eigenvalue and eigenvector errors; orthogonalized variants converge faster per iteration but require growing small generalized eigenproblems [1304.2631].

## 5. Mutual-information-driven RF-chain and parameter selection

In MIMO integrated sensing and communications, GES is a pruning algorithm for RF chains. The objective is a weighted, normalized sum of communication and sensing mutual information:
$$
\frac{\omega_c}{T}I_c(\mathcal{N}_t)+\frac{\omega_s}{N_s}I_s(\mathcal{N}_t),
\qquad
\omega_c+\omega_s=1.
$$
For communication,
$$
I_c(\mathcal{N}_t)
=
T\log_2\left|I_K+\gamma H_c^H(\mathcal{N}_t)H_c(\mathcal{N}_t)\right|
=
T\log_2\left|I_{N_c}+\gamma H_c(\mathcal{N}_t)H_c^H(\mathcal{N}_t)\right|,
$$
while sensing uses
$$
I_s(\mathcal{N}_t)
=
\sum_{n=1}^{N_s}\log_2\left|I_K+\gamma T\,R_{T,n}(\mathcal{N}_t)\right|.
$$
Using determinant-lemma arguments, the total MI is decomposed into per-chain contributions. For a remaining set $\bar{\mathcal{N}}_t$, the chain score is
$$
C_j
=
(1-\gamma\alpha_j)^{\omega_c}
\prod_{n=1}^{N_s}(1-\gamma T\,\beta_{n,j})^{\omega_s/N_s},
$$
with $\alpha_j=h_j^HAh_j$ and $\beta_{n,j}=g_{n,j}^HB_ng_{n,j}$. The greedy update removes the chain with the smallest $C_j$, then updates the relevant inverses by rank-one formulas [2507.09960].

The computational contrast with exhaustive search is explicit. GES has polynomial complexity
$$
O\!\left(N_c^3+N_sN_t^3+(N_t-K)\left(N_c^2+\sum_n r_n^2+N_t\right)\right),
$$
whereas exhaustive search scales as
$$
O\!\left(\binom{N_t}{K}(N_c^3+N_sN_t^3)\right).
$$
In the reported beamspace/hybrid setting with $N_t=16$, $N_c=8$, $N_s=8$, $K=8$, $\omega_c=\omega_s=0.5$, and SNR from $-10$ to $30$ dB, GES and exhaustive-search weighted-MI curves coincide with gap $<0.5\%$, GES remains within $1\%$ of the global optimum across all $K$, and at $30$ dB yields an energy-efficiency gain of about $8\%$ over full selection [2507.09960].

An operator-design analogue appears in One-Way Navier-Stokes (OWNS) methods. There the goal is to choose recursion parameters
$$
\Xi_{N_\beta}=\{\beta_+^j,\beta_-^j\}_{j=0}^{N_\beta-1}
$$
for OWNS-P and OWNS-R approximations to the exact one-way projector. The key objectives are
$$
\mathcal{J}(\Xi)
=
\max_{m\le N_+,\;n>N_+}
\hat{\mathcal{J}}_+(\alpha_m;\Xi)\hat{\mathcal{J}}_-(\alpha_n;\Xi),
$$
and
$$
\mathcal{J}^{(R)}(\Xi)
=
\max\left\{
\max_{m\le N_+}\hat{\mathcal{J}}_+(\alpha_m;\Xi),\;
\max_{n>N_+}\hat{\mathcal{J}}_-(\alpha_n;\Xi)
\right\},
$$
with
$$
\hat{\mathcal{J}}_+(\alpha;\Xi)
=
\prod_j\frac{|\alpha-\beta_+^j|}{|\alpha-\beta_-^j|},
\qquad
\hat{\mathcal{J}}_-(\alpha;\Xi)
=
\prod_j\frac{|\alpha-\beta_-^j|}{|\alpha-\beta_+^j|}.
$$
Because direct combinatorial minimization is intractable, Algorithm 1 greedily identifies the downstream eigenvalue maximizing $\hat{\mathcal{J}}_+$ and the upstream eigenvalue maximizing $\hat{\mathcal{J}}_-$, then appends those as the next pair $(\beta_+^j,\beta_-^j)$ [2506.02320].

The paper gives explicit convergence and stability statements. OWNS-P converges if the downstream/upstream products tend to zero and all downstream and upstream eigenvalues are distinct; OWNS-R has an analogous branchwise condition. If $\beta_+^j=\alpha_m$ exactly, the corresponding downstream mode is retained with zero error, and if $\beta_-^j=\alpha_n$ exactly, the corresponding upstream mode is removed. Numerically, greedy OWNS-P reaches $\mathcal{J}<10^{-8}$ by $N_\beta\approx 30$ in a 2D subsonic flat-plate case, and machine-zero projection-operator error at $N_\beta=29$, whereas the heuristic alternative still has error about $10^{-4}$. In a 3D oblique-wave test, greedy OWNS-P reaches machine-zero error by $N_\beta=24$ rather than roughly $40$ for the heuristic. The same paper also notes an important limitation: for a linear Mack’s second-mode case, the heuristic OWNS-P can converge with $N_\beta=1$ whereas the greedy method needs $N_\beta=6$ [2506.02320].

## 6. Terminological ambiguity, related greedy methods, and recurrent misunderstandings

The supplied literature suggests that a recurring source of confusion is the acronym itself. In causal-discovery work, GES denotes a different greedy DAG-search procedure rather than an eigen-based selection method. That algorithm alternates a forward phase of legal single-arc additions and a backward phase of deletions, and its moves correspond to edges of the characteristic imset polytope $\mathrm{CIM}_p$. More precisely, forward and backward moves are realized as edge traversals, skeleton restrictions correspond to faces $\mathrm{CIM}_G$, and this geometric viewpoint leads to generalized move classes such as turn pairs and edge pairs, as well as the algorithms greedy CIM and skeletal greedy CIM [2103.03771]. This is not an eigen-based formulation, but it is directly relevant to the nomenclature of “GES” in adjacent literatures.

A second misunderstanding is that all GES variants should inherit the same approximation theory. The record is mixed. Subset-selection GES has a $(1-e^{-\gamma})$ guarantee in terms of the submodularity ratio, with a further sparse-eigenvalue lower bound [1102.3975]. Generalized CSS, by contrast, has no proved constant-factor or relative-error bound against the optimal $\ell$-subset, and the existence of a $(1-1/e)$-type guarantee remains open [1312.6820]. Sparse symmetric-eigenvalue GES is controlled by residual-norm and spectral-gap bounds rather than submodularity [1911.10041]. OWNS GES is justified by explicit projector-approximation objectives and stability conditions, not by combinatorial approximation ratios [2506.02320].

A third misunderstanding is that “eigen-based” necessarily implies expensive global spectral computation at every step. The papers do not support that simplification. Forward-regression GES is implemented via Cholesky or inverse updates of $\Sigma_S$ [1102.3975]. Generalized CSS uses precomputed $A^TB$ and rank-one updates of numerator and denominator statistics [1312.6820]. The localized eigenpair method solves reduced eigenproblems and scores complement indices through $B_t^Tx_t$ or a first-order perturbation surrogate [1911.10041]. RF-chain GES removes chains through determinant-lemma-derived contribution scores and rank-one inverse updates [2507.09960]. Even in OWNS, the greedy rule attacks the dominant downstream and upstream eigenvalue branches rather than solving a fresh global optimization problem at each step [2506.02320].

Taken together, these formulations show that GES is best understood as a family resemblance rather than a single algorithm: greedy local updates, spectrally informed scoring, and active-set evolution are the common elements, while the objective, theory, and computational profile depend strongly on the application domain.

Source: https://www.emergentmind.com/topics/greedy-eigen-based-selection-ges