Greedy Eigen-Based Selection (GES)

• GES is a family of greedy algorithms that iteratively select features or parameters based on spectral scores like explained variance, projection error, and residual magnitude.
• It spans applications such as subset selection, column subset selection, localized eigenpair extraction, RF-chain pruning in MIMO systems, and OWNS parameter tuning.
• GES leverages efficient computational strategies, including inverse updates and rank-one recursions, to achieve near-optimal performance with controlled cost.

Greedy Eigen-Based Selection (GES) denotes a class of greedy procedures that iteratively choose, retain, or discard coordinates, columns, atoms, or parameters using a score derived from explained variance, projection error, residual magnitude, mutual information, or a spectral approximation criterion. The supplied literature suggests that the label does not identify a single canonical algorithm. Instead, it appears in several distinct settings: forward regression for subset selection (Das et al., 2011), greedy generalized column subset selection (Farahat et al., 2013), localized eigenpair computation from principal submatrices (Hernandez et al., 2019), RF-chain pruning in MIMO integrated sensing and communications (Shin et al., 14 Jul 2025), and recursion-parameter choice for One-Way Navier-Stokes approximations (Sleeman et al., 2 Jun 2025). Across these settings, the common structure is iterative greedy search guided by a reduced spectral surrogate.

1. Canonical pattern and scope

In the supplied papers, GES follows a recurring template: maintain a current active set, evaluate a per-candidate score, and greedily update the active set so as to improve an objective or reduce a residual. The objective varies by domain, but the update is always local and myopic rather than combinatorial or exhaustive.

Setting	Objective	Greedy action
Subset selection	Maximize $f(S)=b_S^T\Sigma_S^{-1}b_S$	Add the variable with largest marginal gain
Generalized CSS	Minimize $\|B-P_SB\|_F^2$	Add the column with largest error reduction
Sparse eigenvalue computation	Reduce eigenpair residual of a principal submatrix	Add indices with largest residual or perturbation score
MIMO ISAC RF-chain selection	Maximize weighted communication/sensing MI	Remove the chain with smallest contribution score
OWNS recursion design	Minimize projector-approximation objective	Append the worst downstream and upstream eigenvalues

This breadth has two immediate consequences. First, “eigen-based” does not always mean repeated full eigendecompositions; in several formulations the decisive quantity is an inverse-update, a residual, or a projection formula. Second, approximation guarantees are not uniform across the literature: some variants admit explicit bounds, while others remain analytically open.

2. Forward-regression GES for subset selection

The most explicit early formulation appears in the subset-selection problem of Das and Kempe, where one is given zero-mean, unit-variance variables $V=\{X_1,\ldots,X_n\}$ with covariance matrix $\Sigma\in\mathbb{R}^{n\times n}$ and a target variable $Z$ with covariance vector $b\in\mathbb{R}^n$. For any $S\subseteq V$, the explained variance is

$$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$

and the subset-selection problem is to find $S\subseteq V$, $|S|\le k$, maximizing

$\|B-P_SB\|_F^2$0

GES is “essentially ‘Forward Regression’,” selecting at each step the variable with the largest increase $\|B-P_SB\|_F^2$1 (Das et al., 2011).

The analysis centers on the submodularity ratio

$\|B-P_SB\|_F^2$2

together with the smallest $\|B-P_SB\|_F^2$3-sparse eigenvalue

$\|B-P_SB\|_F^2$4

If $\|B-P_SB\|_F^2$5 is the greedy $\|B-P_SB\|_F^2$6-set and $\|B-P_SB\|_F^2$7 is an optimal $\|B-P_SB\|_F^2$8-set, then the recurrence

$\|B-P_SB\|_F^2$9

implies

$V=\{X_1,\ldots,X_n\}$0

Lemma 3.2 further shows $V=\{X_1,\ldots,X_n\}$1, yielding

$V=\{X_1,\ldots,X_n\}$2

A central significance of this result is that the guarantee is controlled by approximate submodularity rather than only by sparse-eigenvalue or RIP-style conditions.

The computational structure is also explicit. With a maintained Cholesky factorization or inverse update of $V=\{X_1,\ldots,X_n\}$3, each candidate marginal gain can be computed in $V=\{X_1,\ldots,X_n\}$4, so each iteration costs $V=\{X_1,\ldots,X_n\}$5 and the total runtime is $V=\{X_1,\ldots,X_n\}$6. Experiments on Boston housing $V=\{X_1,\ldots,X_n\}$7, World Bank indicators $V=\{X_1,\ldots,X_n\}$8, and synthetic Gaussian data $V=\{X_1,\ldots,X_n\}$9, with $\Sigma\in\mathbb{R}^{n\times n}$0 up to $\Sigma\in\mathbb{R}^{n\times n}$1, show that GES achieves near-optimal $\Sigma\in\mathbb{R}^{n\times n}$2 on all sets, within $\Sigma\in\mathbb{R}^{n\times n}$3–$\Sigma\in\mathbb{R}^{n\times n}$4 of the exact optimum; OMP is slightly worse, while oblivious selection and $\Sigma\in\mathbb{R}^{n\times n}$5 relaxation are notably weaker. The paper also reports that $\Sigma\in\mathbb{R}^{n\times n}$6 and condition-number/RIP bounds are often near zero on nearly singular data, whereas the observed submodularity ratio $\Sigma\in\mathbb{R}^{n\times n}$7 is substantially larger, often $\Sigma\in\mathbb{R}^{n\times n}$8–$\Sigma\in\mathbb{R}^{n\times n}$9, and correlates much better with the actual $Z$0 gaps (Das et al., 2011).

3. Generalized column subset selection and sparse approximation

A second important usage concerns generalized column subset selection. Here $Z$1 is the source matrix, $Z$2 is the target matrix, and the goal is to choose $Z$3, $Z$4, minimizing

$Z$5

Equivalently,

$Z$6

The sectioned summary notes that the paper itself “never uses the name GES explicitly,” but the method is a greedy, eigen-inspired column-selection routine for the generalized CSS problem (Farahat et al., 2013).

At iteration $Z$7, the residuals are

$Z$8

with current error $Z$9. Adding column $b\in\mathbb{R}^n$0 reduces the error by

$b\in\mathbb{R}^n$1

If $b\in\mathbb{R}^n$2 and $b\in\mathbb{R}^n$3, then this becomes

$b\in\mathbb{R}^n$4

The algorithm therefore selects

$b\in\mathbb{R}^n$5

where $b\in\mathbb{R}^n$6 and $b\in\mathbb{R}^n$7, and updates these quantities by rank-one recursions derived in Theorem 3. The resulting implementation precomputes $b\in\mathbb{R}^n$8 once, then uses incremental updates rather than recomputing full residual statistics.

Its theory is structurally different from the subset-selection case. The paper proves a projection recursion, $b\in\mathbb{R}^n$9 when $S\subseteq V$0, and an error recursion,

$S\subseteq V$1

However, it “does not prove a constant-factor or relative-error bound vs. the optimal $S\subseteq V$2-subset,” and it remains open whether the greedy rule yields a $S\subseteq V$3 bound or similar. Runtime is $S\subseteq V$4, or $S\subseteq V$5 per step when $S\subseteq V$6 is sparse with $S\subseteq V$7 nonzeros per column. Several specializations are exact: setting $S\subseteq V$8 gives ordinary CSS; taking $S\subseteq V$9 recovers the greedy SVD-based CSS method of Çivril and Magdon-Ismail; and if $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$0 is a single vector $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$1, the procedure becomes Orthogonal Least Squares (Farahat et al., 2013).

4. Localized eigenpair extraction and high-dimensional eigenvalue problems

In large sparse symmetric eigenproblems, GES denotes a method that exploits localization of the target eigenvector. Given $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$2 symmetric, the algorithm maintains an “important” index set $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$3, forms the principal submatrix $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$4, computes its smallest eigenpair

$$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$5

embeds $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$6 into the full space as $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$7, and evaluates the residual

$$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$8

Candidates in the complement $$R^2(Z;S)=1-\mathbb{E}[(Z-\hat y)^2]=b_S^T\Sigma_S^{-1}b_S,$$9 are then scored either by the residual-based quantity

$S\subseteq V$0

or by the perturbation-based quantity

$S\subseteq V$1

and the top $S\subseteq V$2 indices are appended to $S\subseteq V$3 (Hernandez et al., 2019).

The spectral interpretation is precise. By the interlacing theorem for principal submatrices, the reduced eigenvalue satisfies $S\subseteq V$4. Error control is residual-driven:

$S\subseteq V$5

when the target eigenvalue is simple and $S\subseteq V$6 is the spectral gap. The approximation can then serve as an initializer for LOBPCG or Lanczos. Complexity depends on the current active-set size $S\subseteq V$7: solving the dense reduced problem costs $S\subseteq V$8, computing $S\subseteq V$9 costs $|S|\le k$0, and selecting the top $|S|\le k$1 scores costs $|S|\le k$2. The reported examples include nuclear configuration-interaction, many-body-localization spin chains, and California road-network analysis, with speedups ranging from $|S|\le k$3–$|S|\le k$4 against direct LOBPCG, to $|S|\le k$5 faster than full-matrix eigs, to an overall $|S|\le k$6 speedup over full-matrix Lanczos when localization is especially sharp (Hernandez et al., 2019).

A related but more variational line of work concerns high-dimensional eigenvalue problems in tensor-product spaces. Cancès, Ehrlacher, and Lelievre introduce the Pure Rayleigh Greedy Algorithm (PRaGA) and Pure Residual Greedy Algorithm (PReGA), together with orthogonalized versions ORaGA, OReGA, and OEGA. In a Hilbert-space setting with bilinear form $|S|\le k$7 and Rayleigh quotient

$|S|\le k$8

PRaGA chooses

$|S|\le k$9

while PReGA chooses

$\|B-P_SB\|_F^2$00

normalizing after each update. Under compact-embedding and coercivity assumptions, these methods are well posed, satisfy $\|B-P_SB\|_F^2$01, and converge strongly in $\|B-P_SB\|_F^2$02 to an $\|B-P_SB\|_F^2$03-normalized eigenvector. In finite dimension, a Łojasiewicz argument yields either geometric decay $\|B-P_SB\|_F^2$04 when $\|B-P_SB\|_F^2$05 or algebraic decay $\|B-P_SB\|_F^2$06 when $\|B-P_SB\|_F^2$07. Numerical tests on a toy SPD-matrix problem and on buckling modes of a microstructured plate exhibit roughly exponential decay of eigenvalue and eigenvector errors; orthogonalized variants converge faster per iteration but require growing small generalized eigenproblems (Cancès et al., 2013).

5. Mutual-information-driven RF-chain and parameter selection

In MIMO integrated sensing and communications, GES is a pruning algorithm for RF chains. The objective is a weighted, normalized sum of communication and sensing mutual information:

$\|B-P_SB\|_F^2$08

For communication,

$\|B-P_SB\|_F^2$09

while sensing uses

$\|B-P_SB\|_F^2$10

Using determinant-lemma arguments, the total MI is decomposed into per-chain contributions. For a remaining set $\|B-P_SB\|_F^2$11, the chain score is

$\|B-P_SB\|_F^2$12

with $\|B-P_SB\|_F^2$13 and $\|B-P_SB\|_F^2$14. The greedy update removes the chain with the smallest $\|B-P_SB\|_F^2$15, then updates the relevant inverses by rank-one formulas (Shin et al., 14 Jul 2025).

The computational contrast with exhaustive search is explicit. GES has polynomial complexity

$\|B-P_SB\|_F^2$16

whereas exhaustive search scales as

$\|B-P_SB\|_F^2$17

In the reported beamspace/hybrid setting with $\|B-P_SB\|_F^2$18, $\|B-P_SB\|_F^2$19, $\|B-P_SB\|_F^2$20, $\|B-P_SB\|_F^2$21, $\|B-P_SB\|_F^2$22, and SNR from $\|B-P_SB\|_F^2$23 to $\|B-P_SB\|_F^2$24 dB, GES and exhaustive-search weighted-MI curves coincide with gap $\|B-P_SB\|_F^2$25, GES remains within $\|B-P_SB\|_F^2$26 of the global optimum across all $\|B-P_SB\|_F^2$27, and at $\|B-P_SB\|_F^2$28 dB yields an energy-efficiency gain of about $\|B-P_SB\|_F^2$29 over full selection (Shin et al., 14 Jul 2025).

An operator-design analogue appears in One-Way Navier-Stokes (OWNS) methods. There the goal is to choose recursion parameters

$\|B-P_SB\|_F^2$30

for OWNS-P and OWNS-R approximations to the exact one-way projector. The key objectives are

$\|B-P_SB\|_F^2$31

and

$\|B-P_SB\|_F^2$32

with

$\|B-P_SB\|_F^2$33

Because direct combinatorial minimization is intractable, Algorithm 1 greedily identifies the downstream eigenvalue maximizing $\|B-P_SB\|_F^2$34 and the upstream eigenvalue maximizing $\|B-P_SB\|_F^2$35, then appends those as the next pair $\|B-P_SB\|_F^2$36 (Sleeman et al., 2 Jun 2025).

The paper gives explicit convergence and stability statements. OWNS-P converges if the downstream/upstream products tend to zero and all downstream and upstream eigenvalues are distinct; OWNS-R has an analogous branchwise condition. If $\|B-P_SB\|_F^2$37 exactly, the corresponding downstream mode is retained with zero error, and if $\|B-P_SB\|_F^2$38 exactly, the corresponding upstream mode is removed. Numerically, greedy OWNS-P reaches $\|B-P_SB\|_F^2$39 by $\|B-P_SB\|_F^2$40 in a 2D subsonic flat-plate case, and machine-zero projection-operator error at $\|B-P_SB\|_F^2$41, whereas the heuristic alternative still has error about $\|B-P_SB\|_F^2$42. In a 3D oblique-wave test, greedy OWNS-P reaches machine-zero error by $\|B-P_SB\|_F^2$43 rather than roughly $\|B-P_SB\|_F^2$44 for the heuristic. The same paper also notes an important limitation: for a linear Mack’s second-mode case, the heuristic OWNS-P can converge with $\|B-P_SB\|_F^2$45 whereas the greedy method needs $\|B-P_SB\|_F^2$46 (Sleeman et al., 2 Jun 2025).

6. Terminological ambiguity, related greedy methods, and recurrent misunderstandings

The supplied literature suggests that a recurring source of confusion is the acronym itself. In causal-discovery work, GES denotes a different greedy DAG-search procedure rather than an eigen-based selection method. That algorithm alternates a forward phase of legal single-arc additions and a backward phase of deletions, and its moves correspond to edges of the characteristic imset polytope $\|B-P_SB\|_F^2$47. More precisely, forward and backward moves are realized as edge traversals, skeleton restrictions correspond to faces $\|B-P_SB\|_F^2$48, and this geometric viewpoint leads to generalized move classes such as turn pairs and edge pairs, as well as the algorithms greedy CIM and skeletal greedy CIM (Linusson et al., 2021). This is not an eigen-based formulation, but it is directly relevant to the nomenclature of “GES” in adjacent literatures.

A second misunderstanding is that all GES variants should inherit the same approximation theory. The record is mixed. Subset-selection GES has a $\|B-P_SB\|_F^2$49 guarantee in terms of the submodularity ratio, with a further sparse-eigenvalue lower bound (Das et al., 2011). Generalized CSS, by contrast, has no proved constant-factor or relative-error bound against the optimal $\|B-P_SB\|_F^2$50-subset, and the existence of a $\|B-P_SB\|_F^2$51-type guarantee remains open (Farahat et al., 2013). Sparse symmetric-eigenvalue GES is controlled by residual-norm and spectral-gap bounds rather than submodularity (Hernandez et al., 2019). OWNS GES is justified by explicit projector-approximation objectives and stability conditions, not by combinatorial approximation ratios (Sleeman et al., 2 Jun 2025).

A third misunderstanding is that “eigen-based” necessarily implies expensive global spectral computation at every step. The papers do not support that simplification. Forward-regression GES is implemented via Cholesky or inverse updates of $\|B-P_SB\|_F^2$52 (Das et al., 2011). Generalized CSS uses precomputed $\|B-P_SB\|_F^2$53 and rank-one updates of numerator and denominator statistics (Farahat et al., 2013). The localized eigenpair method solves reduced eigenproblems and scores complement indices through $\|B-P_SB\|_F^2$54 or a first-order perturbation surrogate (Hernandez et al., 2019). RF-chain GES removes chains through determinant-lemma-derived contribution scores and rank-one inverse updates (Shin et al., 14 Jul 2025). Even in OWNS, the greedy rule attacks the dominant downstream and upstream eigenvalue branches rather than solving a fresh global optimization problem at each step (Sleeman et al., 2 Jun 2025).

Taken together, these formulations show that GES is best understood as a family resemblance rather than a single algorithm: greedy local updates, spectrally informed scoring, and active-set evolution are the common elements, while the objective, theory, and computational profile depend strongly on the application domain.