On the Slow Convergence to Trivial Solutions of Algorithms for Hard Optimization Problems

Abstract: Hard combinatorial optimization problems, many of which are NP-hard, present fundamental algorithmic challenges. Average-case analysis on random instances has emerged as a powerful framework for understanding typical algorithmic performance beyond worst-case guarantees. A substantial body of work has established negative results: for sufficiently hard instances (often controlled by the underlying graph connectivity/constraints density), no known polynomial-time algorithm can significantly outperform naive heuristics in the double asymptotic limit where both problem size and constraints density tend to infinity. We revisit this picture by studying the finite-size behavior of some optimization algorithms across easy, intermediate, and hard regimes. Through rigorous analysis of large-graph asymptotics combined with numerical experiments on canonical problems (maximum independent set and maximum $K$-SAT), we demonstrate that while algorithms do eventually converge to theoretically predicted bounds, this convergence can be remarkably slow. In the intermediate regime where instances are already highly constrained, local algorithms achieve solutions substantially better than their predicted performance in the high-constraint-density limit. This gap between finite-regime and asymptotic behavior has important practical implications: sophisticated algorithmic design remains crucial even when asymptotic theory predicts inevitable failure.

• The paper rigorously characterizes static degree greedy for maximum independent set and random clause assignment for Max K-SAT using differential-equation and phase-based analyses.
• Finite-size experiments show degree-greedy and exponentially weighted max-product achieve MIS ratios near 0.64, while gains above the 1/2 threshold decay slowly with a fitted exponent of approximately 0.4231.
• The paper demonstrates that asymptotic limits can mislead practical benchmarking because simple local algorithms substantially outperform trivial baselines at realistic graph sizes and constraint densities.

Overview

This paper studies a discrepancy between asymptotic theory and finite-size behavior for local algorithms on hard combinatorial optimization problems, focusing on maximum independent set (MIS) on sparse Erdős–Rényi graphs $G(n,\lambda/n)$ and Max $K$-SAT on random CNF formulas. The central claim is that although classical results predict that simple local algorithms cannot beat trivial baselines—half of the independence number for MIS, and random assignment performance $1-2^{-K}$ for Max $K$-SAT—in the double limit where both size and constraint density diverge, this limit is approached so slowly that at practically relevant densities the algorithms substantially outperform their asymptotic predictions. The paper supports this claim with two rigorous asymptotic characterizations (for static degree greedy on configuration-model graphs and for a "random clause" algorithm on Max $K$-SAT), population dynamics analysis of max-product message passing, and simulations at sizes up to $n=10^5$.

The practical motivation is explicit: neural network-based solvers for combinatorial optimization are necessarily evaluated at finite sizes, where classical baselines still perform well above their asymptotic limits. The paper argues that benchmarking such solvers against asymptotic bounds is misleading.

Algorithms studied

For MIS, three algorithms are analyzed. Static degree greedy (SDG) sequentially selects uniformly at random an unexplored node of minimum degree in the original graph, activating it and blocking its unexplored neighbors. Dynamic degree greedy (DG) is identical except that degrees are recomputed on the induced subgraph of unexplored nodes, making it adaptive. Both terminate after at most $|V|$ steps with a maximal independent set. Max-product (MP) is implemented as the min-sum reparameterization of max-product belief propagation applied to a weighted MIS formulation as a MAP inference problem in a binary pairwise Markov random field. A key methodological contribution is a modification of MP: node weights are set to $w_i = \exp(-|\partial i|)$, i.e., exponentially decaying in degree. This biases the algorithm toward low-degree nodes and empirically improves both convergence and performance; notably, the improvement holds only for weights with at least exponential decay—polynomially decaying weights were observed to hinder convergence.

For Max $K$-SAT, the paper introduces the random clause (RC) algorithm: starting from the empty assignment, it repeatedly picks a uniformly random undecided clause, then a uniformly random unassigned literal within it, and sets that variable to satisfy the literal. It treats all undecided clauses uniformly regardless of remaining length, running for exactly $n$ steps.

Asymptotic characterizations

Two theorems provide rigorous large-graph limits. Theorem 1 concerns SDG on the configuration model $K$0 with empirical degree distribution converging weakly to $K$1 (with second-moment convergence). Using a phase decomposition—phase-$K$2 exhausts all remaining unexplored nodes of degree $K$3—and Wormald-style differential equation analysis under deferred decisions, the theorem shows that the fraction of nodes in the SDG independent set converges in probability to $K$4, where the $K$5 are given by explicit recursions over auxiliary sequences $K$6 tracking unexplored nodes and half-edges across phases. The proof embeds the exploration in continuous time with activation rate $K$7 per degree-$K$8 node during phase $K$9.

Theorem 2 gives the analogous result for RC: the satisfied-clause proportion $1-2^{-K}$0 converges in probability to $1-2^{-K}$1, where $1-2^{-K}$2 solve a closed ODE system driven by three drift mechanisms (selection of the chosen clause, removal or satisfaction of clauses containing the assigned variable, and length reduction of clauses whose literal is falsified), stopped when the undecided-clause density $1-2^{-K}$3 vanishes. The proof via the differential equation method is simpler than the SDG case because RC has no phases and needs no continuous-time embedding.

Both theorems are validated numerically: finite-size simulations ($1-2^{-K}$4 for MIS; $1-2^{-K}$5 for Max $1-2^{-K}$6-SAT) agree closely with the theoretical curves.

Main empirical findings

The headline result concerns the performance ratio $1-2^{-K}$7, using the Coja-Oghlan–Efthymiou estimate $1-2^{-K}$8 for $1-2^{-K}$9. Existing conjectures based on the overlap gap property assert that no polynomial-time local algorithm achieves a ratio exceeding $K$0 asymptotically. Against this backdrop:

Algorithm	Regime	Reported behavior
DG	$K$1, $K$2	Ratio approaches ≈ 0.64, well above 1/2
SDG	same	Above 1/2; matches its own asymptotic prediction
MP (with exponential weights)	$K$3, $K$4	Above 1/2; closely matches SDG's curve
MP population dynamics	large graph limit	Confirms finite-size MP behavior

Two findings deserve emphasis. First, fitting the relative gain $K$5 above the half-threshold to a power law $K$6 over $K$7 yields a decay exponent $K$8—a quantitatively slow approach to the asymptotic bound, with non-negligible gains persisting even at $K$9. Second, the near-coincidence of MP and SDG performance is explained structurally: maximizing $K$0 induces, in the fast-decay limit, a hierarchical optimization that saturates the minimum-degree class first—precisely SDG's exploration philosophy—so the two algorithms from different families converge to nearly identical targets despite different mechanisms.

Heatmaps over $K$1 show the performance gain persists essentially unchanged across these sizes for sparse instances, though ratios fall below 1/2 for denser graphs.

Extension to Max K-SAT

The parallel phenomenon appears for RC. For fixed variable $K$2 at high clause density $K$3, positive and negative appearances are balanced up to lower-order terms, so assignment decisions become effectively neutral and RC's performance converges to the random-assignment baseline $K$4 (e.g., $K$5 for $K$6). Simulations and ODE predictions agree well even at $K$7, and the convergence to the baseline is monotone but slow on a logarithmic scale of $K$8, with a broad intermediate range of densities where RC retains a meaningful advantage over random assignment. The qualitative structure thus mirrors MIS: eventual collapse to the trivial solution, approached slowly.

Limitations and open questions

Several caveats are stated plainly in the paper. The optimality of degree-greedy for $K$9 remains unproven, resting on conjecture and numerical evidence. The asymptotic analysis covers only SDG, not DG; tracking the full evolving degree distribution of the induced subgraph $n=10^5$0 is left open. The claimed barrier—that no polynomial-time algorithm exceeds half-optimality—is a conjecture, not a theorem, so the observed above-threshold performance does not contradict any proven impossibility result. The exponential-weight modification of MP is justified empirically rather than through a convergence guarantee on loopy graphs, and the restriction to exponentially (not polynomially) decaying weight functions lacks a formal explanation. For Max $n=10^5$1-SAT, the slow-convergence claim is established for RC specifically; whether stronger algorithms exhibit the same finite-size advantage at high density is not addressed. Finally, the power-law fit exponent $n=10^5$2 is numerical rather than derived analytically.

Conclusion

The paper establishes, both rigorously and numerically, that the asymptotic collapse of simple local algorithms to trivial solutions on hard random optimization problems occurs too slowly to govern practical regimes: SDG, DG, and appropriately weighted max-product all exceed the conjectured half-optimality threshold on Erdős–Rényi graphs at large-but-finite mean degree, with a fitted decay exponent near 0.42, and the analogous gap appears for the random clause algorithm on Max $n=10^5$3-SAT. The main implication is methodological: since neural solvers are evaluated at finite sizes, the appropriate baseline is not the asymptotic bound but the finite-size performance already attainable by simple degree-biased local algorithms—a standard the paper suggests existing benchmarking efforts should adopt.