---
title: Greedy Algorithm for Basis Selection
url: https://www.emergentmind.com/topics/greedy-algorithm-of-a-basis
type: topic
---

# Greedy Algorithm for Basis Selection

A greedy algorithm of a basis refers to a selection or approximation procedure that iteratively chooses elements, coordinates, or vectors so as to optimize a specific cost function at each step—most often the reduction of a norm, error, or residual—according to the largest available quantity (e.g., coefficient modulus, error estimate, or stabilizer shrinking). Greedy algorithms are fundamental in Banach space theory, computational approximation, permutation group bases, cluster algebras, and reduced order modeling. Their effectiveness and theoretical understanding depend crucially on the geometric properties of the underlying basis and ambient space.

## 1. Thresholding Greedy Algorithm and Basic Structures

Let \( X \) be a Banach (or quasi-Banach) space equipped with a (semi-normalized) Markushevich or Schauder basis \( \{e_n\} \) and biorthogonal functionals \( \{e_n^*\} \). Every \( x \in X \) admits an expansion \( x = \sum_{n=1}^\infty e_n^*(x) e_n \).

The **Thresholding Greedy Algorithm (TGA)** proceeds as follows. For a given \( x \), enumerate the coefficients \( |e_n^*(x)| \) in non-increasing order, producing a greedy ordering. The \( m \)-term greedy approximant is then
\[
G_m(x) := \sum_{n \in \Lambda_m(x)} e_n^*(x) e_n,
\]
where \( \Lambda_m(x) \) is a set of \( m \) indices with the largest \( |e_n^*(x)| \). The algorithm thus selects the "most significant" basis elements for the current approximation.

An **\( m \)-term best approximation error** is defined as
\[
\sigma_m(x) := \inf_{\substack{A \subset \mathbb N,\,|A|=m \\ a_n \in \mathbb F}} \left\| x - \sum_{n \in A} a_n e_n \right\|.
\]

Greedy algorithms exist in parametric PDE model reduction, operator reconstruction, permutation group base computations, and algebraic bases (cluster, Lie, etc.).

## 2. Greedy-Type Bases: Classification and Characterization

**Greedy bases** are those for which the TGA achieves (up to a multiplicative constant) the best possible approximation of any vector by sums over the basis. The canonical characterization (Konyagin–Temlyakov [2405.20939]) is
\[
\|x - G_m(x)\| \leq C\, \sigma_m(x),
\]
for all \( x \) and \( m \), with the smallest constant \( C = C_g \). The essential equivalences are:

- Greedy \( \Leftrightarrow \) unconditional (suppression of coordinates does not increase the norm) + democratic (norms of blocks of equal length are comparable).

Other key types include:

- **Quasi-greedy:** greedy approximants are uniformly bounded in norm, i.e., \( \| G_m(x) \| \leq C \| x \| \).
- **Almost greedy:** TGA error is comparable to the best projection error onto any coordinate subspace, characterized as quasi-greedy + democratic.
- **Strongly partially greedy:** TGA error is compared to partial sum errors, characterized by quasi-greedy + conservative properties.
- **Weighted, f-greedy, and sequential analogues:** Introduced through extensions with weights, slow-growing functions, or specific index selection rules, modifying classical democracy and unconditionality requirements [1803.02932, 2209.09628, 2302.05758].

Weighted variants, e.g., \( w \)-greedy, replace block sizes by sums of weights \( w(A) \), and require \( w \)-democracy for equivalence.


## 3. Summability Methods and Advanced Convergence Properties

Beyond raw greedy sums, **summability methods** use averages of greedy approximants to define new basis classes:

- **Cesàro-quasi-greedy (CQG):** Cesàro means \( \sigma_n(x) = \frac{1}{n} \sum_{k=1}^n G_k(x) \) are uniformly bounded.
- **de la Vallée Poussin-quasi-greedy (VPQG):** Means \( V_n(x) = \frac{1}{n+1} \sum_{k=0}^n G_{n+k}(x) \) are bounded.

Main equivalences proved in [2504.13350] include:
- CQG \(\Leftrightarrow\) norm convergence of Cesàro means for all greedy orderings.
- VPQG \(\Leftrightarrow\) norm convergence of VP means to greedy approximants.

VPQG and CQG bases are shown to be nearly unconditional, possess partial unconditionality for large coefficients, and admit strong subsequence convergence phenomena.

## 4. Extension to Quasi-Banach Spaces and Further Generalizations

In quasi-Banach spaces (\(0 < p \leq 1\)), greedy-type approximation theorems require stabilization and non-convexity tools. The operator form and democracy-type properties are preserved, but renorming, block decompositions, and truncation arguments must be handled via the Aoki-Rolewicz lemma and density of finitely supported vectors [1903.11651]. Analogues of the thresholding greedy algorithm exist for Markushevich bases, with precise conditions for quasi-greediness, democracy, and almost-greediness (cf. LUCC, SUCC properties).

Recent work further generalizes:
- Greedy bases via constant-coefficient polynomials and configuration pairings (left-greedy bases in Lie algebra theory) [1510.06984, 2309.14607].
- The relationship of greedy approximants to maximal inequalities and order convergence in Banach lattices [2601.01047].

## 5. Adaptive, Reduced Order, and Batch Greedy Algorithms

Greedy algorithms play a pivotal role in reducing the dimensionality of high-dimensional function spaces encountered in parametric PDEs and operator identification, notably via reduced basis methods.

- Classical greedy and **weak greedy algorithms** use a posteriori error estimators for snapshot selection, with rigorous convergence rates linked to Kolmogorov widths [2407.11631, 1911.02214].
- **Parallel batch greedy algorithms** (batch size \( b \)) allow for multiple snapshot selection per iteration, leading to exponential error decay rates nearly independent of \( b \), with significant computational savings as demonstrated in thermal block model problems.
- **Natural greedy algorithms (NGA)** in Banach spaces utilize norming functionals to project onto subspaces recursively, avoiding costly projections and maintaining convergence comparable to orthogonal greedy algorithms in Hilbert spaces [1905.06448].
- **Optimized Greedy Reconstruction (OGR)** adaptively selects basis elements that improve identifiability in operator reconstruction, particularly in quantum mechanics and control theory [2010.16130].

Hybrid and adaptive methods further accelerate offline training by decomposing the training set, enriching active pools, or reconstructing surrogate error metrics [1911.02214].

## 6. Structural Properties, Hierarchies, and Open Problems

Characterizations of greedy-type bases hinge on structural invariants:
- **Democracy:** Uniform norm comparability for unit-coordinate blocks.
- **Unconditionality:** Norm stability under coordinate removal.
- **Symmetry for largest coefficients, Property (A):** Uniform behavior under sign-changes and block exchanges.
- **Conservativity:** Norm control under index set gaps.
- **Sequential and interval-support properties:** Greedy approximation performance with index sets restricted to intervals or arithmetic progressions.

Many open problems remain:
- Isometric renorming to 1-greedy status (especially Haar basis in \(L_p\), \(p \neq 2\)) [2405.20939].
- Characterization of quasi-greedy basic sequences in all Banach spaces.
- Equivalence between summability-based greedy classes and classical quasi-greedy bases [2504.13350].
- The precise implications of democracy and super-democracy for weighted/generalized greedy bases.

## 7. Illustrative Examples

| Basis/Class                          | Norm Structure or Specialization   | Greedy Characterization/Property |
|--------------------------------------|------------------------------------|----------------------------------|
| Canonical ℓₚ basis (1 ≤ p < ∞)      | Unconditional, equi-norm           | 1-greedy, weighted/constant      |
| Haar system in Lₚ (1 < p < ∞)       | Unconditional, democratic          | Greedy, but renorming open       |
| Weighted spaces with decaying w      | Varied democracy/unconditionality  | Weighted greedy/weighted Property (A) |
| ℓₚ ⊕ ℓ_q basis (1 < p < q < ∞)      | No democratic basis                | f-greedy, e.g., for f(m) = m^{p/q} [2209.09628] |
| Compact wavelet bases in Lₚ          | Maximal inequality holds           | Order and uniform convergence    |
| Interval-supported approximations    | Consecutive greedy, almost greedy  | Equivalent in Schauder case [2302.05758] |

These examples demonstrate the interplay of geometric structure and greedy algorithm performance in high-dimensional and function-theoretic contexts.

---

The modern theory of greedy algorithms of a basis synthesizes combinatorial selection rules, geometric invariants, and operator-theoretic properties. Its ramifications are evident in nonlinear approximation, computational group theory, representation theory, functional analysis, and reduced order modeling. The ongoing development focuses on elucidating precise renorming conditions, summability implications, and the extension of greedy properties to exotic or abstract Banach and quasi-Banach spaces.

Source: https://www.emergentmind.com/topics/greedy-algorithm-of-a-basis