---
title: Core-Based Greedy Algorithms
url: https://www.emergentmind.com/topics/core-based-greedy-algorithms
type: topic
---

# Core-Based Greedy Algorithms

Core-based greedy algorithms refer to a class of methods that construct small, representative summaries (“core-sets”) of large datasets using greedy selection rules, with the aim of preserving optimization objectives of interest under composable or distributed frameworks. These algorithms have gained prominence in settings such as determinant maximization, column subset selection, and clustering with outliers, especially where parallelization, streaming, and scalability are required. They enable effective approximation guarantees by combining the structure of greedy algorithms with provably composable summaries, often exhibiting strong empirical performance and theoretical bounds in both centralized and distributed regimes.

## 1. Composable Core-sets: Foundations and Definitions

A composable core-set is a mapping $c$ assigning to any dataset $P$ a small subset $c(P) \subseteq P$, such that for any collection $\{P_i\}$, evaluating the global objective on the merged core-sets $\bigcup_i c(P_i)$ achieves a constant-factor approximation (or better) to the same objective on the full union $\bigcup_i P_i$. Formally, for an $\alpha$-composable core-set, it holds that for all $\{P_i\}$,
$$
\mathrm{OPT} \left( \bigcup_i c(P_i) \right) \geq \frac{1}{\alpha} \cdot \mathrm{OPT} \left( \bigcup_i P_i \right)
$$
where $\mathrm{OPT}(\cdot)$ is the optimization objective, and $|c(P)| \ll |P|$ is desirable for computational and communication efficiency.

Randomized composable core-sets extend this to randomized constructions and expectations, particularly in distributed environments where data is partitioned and local summaries are aggregated. The key property is that good local coverage accumulates globally after union and an optional further greedy or local search refinement, enabling near-linear distributed computation for otherwise intractable objectives [1605.08795].

## 2. Greedy Algorithms for Determinant Maximization and DPP MAP Inference

In determinant maximization (MAXDET$_k$), given $P \subset \mathbb{R}^d$ and $k$, the goal is to select $S \subset P$, $|S|=k$, maximizing $\det(V_S V_S^\top)$, which encodes the squared volume of the parallelepiped spanned by $S$. This problem coincides with MAP inference for $k$-DPPs when the kernel is $L=VV^\top$.

A greedy approach initializes $C = \emptyset$ and iteratively appends the point $p$ maximizing its distance from $\mathrm{span}(C)$, repeating $k$ times. This strategy preserves a geometric “directional height” guarantee: for any $(k-1)$-subspace $H$, $h(C,H) \geq h(P,H)/(2k3^k)$. Via a reduction (Corollary 3.2) any $\alpha$-approximate coreset for $k$-directional height yields an $\alpha^{2k}$-composable core-set for MAXDET$_k$.

The greedy core-set thus provides an $O(C^{k^2})$ approximation, where $C$ is $O(k3^k)$. A subsequent local search algorithm—starting from the greedy solution and iteratively performing improving swaps—achieves a tighter $O(k^{2k})$ bound, with improved directional height preservation $h(C,H) \geq h(P,H)/(2k(1+\epsilon))$. These methods are practical and more memory/computation efficient than prior LP-based constructions, while achieving strong empirical and theoretical guarantees on standard datasets [1907.03197].

| Algorithm      | Core-set Size | Approx. Factor | Principle            |
| -------------- | ------------- | -------------- | ------------------- |
| Greedy         | $k$           | $O(C^{k^2})$   | Span-maximization   |
| Local Search   | $k$           | $O(k^{2k})$    | Swaps for det. gain |
| LP-based (ref) | $O(k\log k)$  | Near-optimal   | Linear programming  |

The effectiveness of these approaches is validated by experimental results where Local Search consistently improves determinant values compared to Greedy (offline LS vs. GD, 5–13% higher; as core-sets, 1.9–9.6% improvement), at moderate additional runtime [1907.03197].

## 3. Greedy Core-Set Methods in Column Subset Selection

Column Subset Selection (CSS) involves selecting $k$ columns from $A \in \mathbb{R}^{m \times n}$ maximizing the explained variance $\| \Pi_{A[S]} A \|_F^2$, with $\Pi_{A[S]}$ the projector onto $\mathrm{span}\{A_j : j \in S\}$. The standard greedy (GCSS) iteratively picks the column offering the greatest incremental explained variance. Improved analysis yields a guarantee dependent only on the condition number $\kappa$ of the optimal $k$-subset, as opposed to the worst-case over all $k$-subsets.

Composable randomized greedy core-sets for CSS operate by partitioning data, running local greedy algorithms with “overshoot” (selecting $O(k/\sigma_{\min})$ columns per partition), merging summaries, then performing a final greedy pass to extract $k$ columns. The guarantee is that, in expectation, the output achieves at least $1/(8\kappa)$ of the optimum, which can be boosted to $(1-\varepsilon)$-approximation with $O(\kappa/\varepsilon)$ passes [1605.08795].

Experimentally, distributed greedy (DistGreedy) methods match or outperform classic baselines (e.g., SVD-based 2-Phase, PCA) in explained variance and downstream classification, particularly in high-dimensional or sparse settings, with 10–70$\times$ speedups on large datasets [1605.08795].

## 4. Core-based Greedy Algorithms for $k$-Center Clustering with Outliers

For the $k$-center with $z$ outliers problem, the objective is to select $k$ centers $C$ and remove $z$ points to minimize the maximum cluster radius over the non-outliers. Greedy core-set-based algorithms here are variants of Gonzalez’s algorithm, adapted for outlier robustness.

The bi-criteria greedy approach constructs a set $E$ by:
1. Randomly sampling initial points,
2. Iteratively identifying the $(1+\epsilon)z$ furthest points to the current set,
3. Sampling a small batch from these and adding them to $E$.

After $O(k/\epsilon)$ rounds, $E$ (of size $O(k/\epsilon)$) provides a $(2,O(1/\epsilon))$-approximation in the relaxed $(k,z)_\epsilon$-center sense, w.h.p. [1901.08219][2301.02814]. Single-criterion versions exist for $k=O(1)$ with similar approximation.

Coreset construction in doubling metrics leverages the covering property: the inlier subset with doubling dimension $\rho$ can be covered by $O((2/\mu)^\rho k)$ balls of radius $\mu r_{\mathrm{opt}}/2$. The core-set comprises a weighted summary of representatives plus the furthest $2z$ points, achieving $\mu r_{\mathrm{opt}}$ additive error for any solution.

The resulting core-set size is $2z + O((2/\mu)^\rho k)$, enabling orders-of-magnitude reductions in downstream optimization time (5–30$\times$ faster, preserving $\leq1\%$ loss in radius at $|E| \lesssim 10\% n$). Distributed compositions use two communication rounds with $O(z + k(2/\mu)^\rho\log^2(1/\eta))$ points per site [2301.02814].

| Variant      | Core-set Size         | Approx. Factor | Metric Requirements    |
| ------------ | -------------------- | -------------- | --------------------- |
| Bi-criteria  | $O(k/\epsilon)$      | $(2,O(1/\epsilon))$ | General metrics       |
| Doubling-core| $2z+O((2/\mu)^\rho k)$ | Additive $\mu r_{\mathrm{opt}}$ | Doubling dim. $\rho$ |

## 5. Computational Complexity and Scalability

Core-based greedy algorithms are designed for strong computational efficiency, particularly as data cardinality or dimension grows.

- **Determinant maximization**: Greedy runs in $O(n k^2 d)$ total time, with $O(nd)$ storage; Local Search incurs higher worst-case cost $O(n k^4 \log k / \epsilon)$; kernelized versions require only inner-product queries [1907.03197].
- **CSS**: Standard greedy and its distributed variants exploit partitioned data and parallelism, with communication proportional to total core-set size, and critical dependence on the minimum singular value of the optimal subset [1605.08795].
- **$k$-center with outliers**: Each round of the greedy clustering costs $O(n)$ or $O(n D)$ in Euclidean $\mathbb{R}^D$, total $O((k/\epsilon) n D)$. Coreset construction in doubling metrics is $O(n (2/\mu)^\rho k \log^2(1/\eta))$; sublinear variants avoid dependency on $n$ [2301.02814][1901.08219].

Empirically, all surveyed algorithms demonstrate speed and memory advantages over global optimization or LP-based regimes, with the core-set approach yielding particular gains in downstream problem-solving stages.

## 6. Theoretical Guarantees and Proof Techniques

Core-based greedy algorithms leverage a range of theoretical foundations:
- **Directional height preservation** enables reductions from geometric coverage objectives to volume/determinant maximization, with tight approximation factors analyzable via inductive and projection arguments [1907.03197].
- **Martingale concentration** and **Azuma–Hoeffding** inequalities are used in $k$-center analyses to show rapidly covering all optimal clusters with high probability [2301.02814][1901.08219].
- **Random partitioning** analysis ensures that composable core-sets accumulate global coverage in distributed or streaming models; oversampling via “overshoot” parameters guarantees that optimal solutions are not lost during partition-local selection [1605.08795].
- Core-set size and approximation guarantees for doubling metrics are obtained via successive covers: each optimal cluster is covered by bounded-size sets whose union forms the compressive summary, with error controlled additively or multiplicatively in the doubling dimension [2301.02814][1901.08219].

A plausible implication is that such structural proof techniques could be generalized to other non-submodular or high-dimensional objectives, leveraging metric or spectral properties unique to problem classes.

## 7. Experimental Evaluation and Applications

Extensive empirical studies across determinant maximization [1907.03197], column subset selection [1605.08795], and $k$-center with outliers [2301.02814][1901.08219] demonstrate:

- Consistent improvements in solution quality (e.g., determinant maximization, $k$-center radius) versus baseline greedy or LP methods at competitive or lower runtime.
- Strong scaling in high-dimensional and sparse regimes, with distributed core-set aggregation matching or exceeding centralized solution quality with near-linear cost.
- Robustness to outliers and flexibility in real-world data, with summary sizes as low as 10–25% of dataset size preserving objective value within $1\%$.
- Practicability in both synthetic and real datasets, including MNIST, GENES, news20.binary, Shuttle, Covertype, KDD-99, and Poker.

These findings indicate that core-based greedy algorithms are well-suited for large-scale, distributed, and high-dimensional data analysis tasks wherever composable summaries with strong theoretical guarantees are required.

Source: https://www.emergentmind.com/topics/core-based-greedy-algorithms