Greedy Algorithms with Provable Guarantees

• Greedy algorithms with provable guarantees are iterative optimization methods that select locally optimal choices and provide theoretical performance bounds based on properties like submodularity and curvature.
• They leverage parameters such as the submodularity ratio and generalized curvature to ensure solution quality across diverse settings including matroid, partition, and adaptive constraints.
• Variants like Sample Greedy and Differentiable Greedy enhance practical efficiency, making these methods vital for applications in resource allocation, sensor placement, and online learning.

A greedy algorithm with provable guarantees is an iterative optimization strategy that builds a solution step by step, each time making the locally optimal choice under some criterion, and crucially, comes equipped with rigorous worst-case or average-case performance bounds that connect its output to the global optimum. In contrast to heuristic or empirically-motivated greedy procedures, these algorithms rely on precise structural properties of the objective function or combinatorial constraint set—such as submodularity, curvature, or matroid structure—to establish provable approximation or regret ratios. Greedy algorithms with guarantees are foundational in combinatorial optimization, submodular maximization, resource allocation, online learning, and network analysis, and their theoretical understanding has been refined and extended in a variety of directions in the literature.

1. Structural Foundations: Submodularity, Curvature, and Generalizations

Submodularity, defined via diminishing returns, has been central to early greedy approximation analyses. If $f: 2^V \to \mathbb{R}_+$ is monotone, submodular, and normalized ($f(\emptyset) = 0$), classical results guarantee that the greedy algorithm for cardinality-constrained maximization achieves a $(1-1/e)$ approximation factor, matching lower bounds under standard oracle models. For matroid constraints, the greedy method achieves a $1/2$-approximation.

Recent work has extended provable guarantees to settings without full submodularity by introducing parameters such as the submodularity ratio $\gamma$ and (generalized) curvature $\alpha$ (Bian et al., 2017, Summers et al., 2017, Gatmiry et al., 2018). For a monotone, non-submodular objective, $\gamma \in [0,1]$ quantifies the weakest ratio such that

$$\sum_{\omega \in \Omega \setminus S} [f(S \cup \{\omega\}) - f(S)] \geq \gamma [f(S \cup \Omega) - f(S)]$$

for all $S,\Omega \subseteq V$. Generalized curvature $\alpha$ captures the maximum marginal gain drop as elements are added, with lower values of $\alpha$ indicating “closer to modular” behavior. When both parameters can be bounded, the greedy solution $S^K$ with $K$ elements satisfies (Bian et al., 2017, Summers et al., 2017): $$f(S^K) \geq \frac{1}{\alpha}(1-e^{-\alpha \gamma}) f(\mathrm{OPT})$$ This bound recovers the $(1-1/e)$ guarantee for $(\gamma,\alpha) = (1,1)$ and degrades gracefully otherwise. These structural parameters enable meaningful approximation statements even for highly non-submodular but monotone objectives, a scenario common in applications such as Bayesian experimental design, controllability metrics, and LPs with combinatorial constraints.

2. Matroid and Partition Constraints: Beyond Cardinality

Multiple lines of research have analyzed greedy maximization under matroid or partition matroid constraints for both submodular and non-submodular objectives. When $f$ is monotone submodular, classical analysis gives $1/2$-approximation for arbitrary matroids (Gatmiry et al., 2018). For partition matroids, curvature-based analyses yield instance-adaptive bounds: for curvature $\alpha$,

$$f(S_\mathrm{greedy}) \geq \frac{1}{\alpha}(1-\exp(-\alpha\cdot \frac{\overline{d}}{d})) f(\mathrm{OPT})$$

where $d$ is the total allowed size and $\overline{d}$ the smallest individual part bound. For monotone subadditive objectives (not even submodular), the guarantee degrades but remains constant under bounded curvature (Friedrich et al., 2018). Greedy remains computationally efficient, requiring $O(dn)$ value oracle queries.

In general matroid settings with non-submodular objectives but submodularity ratio $\gamma$ or curvature $\alpha$ bounded, greedy achieves (Gatmiry et al., 2018): $$f(\mathrm{greedy}) \geq 0.4\frac{\gamma^2}{\sqrt{\gamma r}+1}f(\mathrm{OPT}) \qquad\text{or}\qquad f(\mathrm{greedy}) \geq \left(1+\frac{1}{1-\alpha}\right)^{-1}f(\mathrm{OPT})$$ where $r$ is the matroid rank. These results describe precise degradation in approximation guarantee as a function of both constraint strength and non-submodularity.

3. Recent Analytical Tools: Discriminant, Sharpness, and Weak Submodularity

Recent analyses have introduced new local instance-dependent tools. The discriminant $d_i$ at step $i$ of greedy is the ratio of the actual chosen incremental gain to the best alternative eligible choice. If the discriminant is much larger than $1$ for all steps, the greedy solution can be provably close to optimal even if the worst-case curvature is nonzero (Rajaraman et al., 2018): $$\frac{f(\mathrm{greedy})}{f(\mathrm{OPT})} \geq \min\left\{1, \frac{1}{c + 1/d_{\min}}\right\}$$ where $c$ is curvature. This localizes the guarantee to actual greedy choices, explaining superior empirical performance in “unbalanced” instances.

The sharpness property (inspired by strong convexity) measures how much marginal gain remains when a solution differs from optimal. If $f$ is $(c,\theta)$-sharp, the greedy solution achieves

$f(S^{g}) \geq [1-(1-\theta/c)^{1/\theta}]f(S^*)$

which can be strictly better than $1-1/e$ in practice (Torrico et al., 2020). This framework explains why greedy often outperforms its worst-case guarantee in real-world applications where the optimal solution is pronounced.

4. Beyond Monotonicity: Non-monotone and Adaptive Settings

For non-monotone submodular maximization, simple or stochastic greedy variants can still achieve constant-factor guarantees efficiently. For instance, a modified stochastic greedy algorithm achieves an expected $1/4$-approximation with linear number of value oracle queries under cardinality constraints, even when $f$ is not monotone (Sakaue, 2019).

Adaptive settings, where decisions and their marginal gains depend on observed outcomes, can be handled using the adaptive submodularity ratio. Even when a function is not adaptive submodular (i.e., the “diminishing returns” property does not hold exactly for adaptive strategies), as long as the adaptive submodularity ratio $\gamma_{\psi,k}$ is bounded away from zero, greedy enjoys

$$\mathrm{avg}(\pi_{\mathrm{greedy}}) \geq [1-e^{-\gamma_{\psi,k}\ell/k}]\,\mathrm{avg}(\pi^*)$$

where $\pi^\ast$ is the optimal adaptive policy (Fujii et al., 2019). This opens greedy methods to a much wider class of adaptive stochastic optimization problems.

5. Limitations, Hardness, and Interpretations

Despite strong practical and theoretical performance in many settings, greedy algorithms with provable guarantees have sharp limitations. For matching problems, MinGreedy cannot improve upon a $1/2$ best-case approximation on adversarial instances, and this ratio is tight even for bipartite graphs; however, empirical results on random graphs show performance much closer to optimal matching (Besser et al., 2015). For centrality minimization in networks, the basic greedy approach may have an approximation ratio no better than $O(|V|)$, but other scalable approximation algorithms leveraging structural graph properties or continuous relaxations (e.g., via Lovász extension) provide substantially improved guarantees in both theory and practice for harmonic centrality (Miyauchi et al., 12 Feb 2024).

In online and learning settings, pure greedy (“greedy index”) algorithms can provably achieve sublinear regret only under additional randomness or structural assumptions (e.g., with arm subsampling in multi-armed bandits (Jedor et al., 2021), or under robust input models in online fair allocation (Yang et al., 2023)).

6. Algorithmic Variants: Sample Greedy, Repeated Greedy, and Differentiable Greedy

To address constant-factor bottlenecks, algorithmic variants build on classical greedy. Sample Greedy, applicable for $k$-extendible systems, first downsamples the ground set before applying greedy, resulting in reduced computation and similar approximation guarantees (up to $k+3$ for non-monotone submodular functions) (Feldman et al., 2017). Repeated Greedy restarts greedy on disjoint subsets to avoid local optima in nonmonotone settings, achieving near-optimal performance with $O(n r \sqrt{k})$ value queries.

For end-to-end learning and decision-focused optimization, differentiable greedy algorithms replace each discrete argmax by a softmax-based selection over the feasible set (Sakaue, 2020). This “smooths” the forward path such that unbiased gradient estimators can be computed, and the classic $(1-1/e)$ approximation guarantee is preserved up to a term controllable by the smoothing parameter.

The recent framework of randomized greedy neural network optimization (using orthogonal greedy algorithm with a randomized discretized dictionary) demonstrates that optimal convergence orders for function approximation or PDEs can be achieved with high probability, with drastic reductions in computational demands (Xu et al., 25 Jul 2024).

7. Applications and Impact in Theory and Practice

Greedy algorithms with provable guarantees are deployed in machine learning (feature selection, experimental design), network science (actuator/sensor placement, centrality minimization), revenue management, online fair division, and resource allocation under matroid and partition constraints. They underpin scalable solutions to massive-scale optimization problems, offer interpretable selection processes (crucial for fair division and market design (Yang et al., 2023)), and, when analyzed through the refined lens of submodularity ratio, curvature, discriminant, or sharpness, yield instance-specific as well as worst-case guarantees that elucidate both typical and adversarial-case performance.

The growth of bicriteria approximation algorithms, randomized variants, and differentiable analogues further extends the reach of greedy methods, providing pathways to multi-agent distributed planning under limited information (Biggs et al., 2022), to robust performance in online learning (Niazadeh et al., 2021), and to decision-differentiable deep learning pipelines. Their versatility and rigor make greedy algorithms with provable guarantees a central component in both combinatorial optimization theory and large-scale algorithmic practice.