HeurAgenix: LLM-Driven CO Heuristic Framework

• HeurAgenix is a two-stage hyper-heuristic framework that uses LLMs to automatically evolve and select heuristic strategies for challenging CO problems.
• It employs contrastive mutation paired with real-time Monte Carlo evaluation to iteratively refine heuristics and reduce solution costs.
• Validated on problems such as TSP, CVRP, MKP, JSSP, and MaxCut, it significantly narrows optimality gaps, achieving near state-of-the-art performance.

HeurAgenix is a two-stage hyper-heuristic framework designed to automate the creation and deployment of heuristics for combinatorial optimization (CO) problems by leveraging LLMs. The framework targets canonical tasks such as the Traveling Salesman Problem (TSP), Capacitated Vehicle Routing Problem (CVRP), Multiple Knapsack Problem (MKP), Job-Shop Scheduling Problem (JSSP), and MaxCut, where the exponential size of the search space renders exact methods infeasible and the adaptation of hand-crafted heuristics across diverse instances remains a major bottleneck. By combining a contrastive heuristic evolution loop with a real-time, LLM-guided selection mechanism, HeurAgenix achieves state-of-the-art results, matching or surpassing both LLM-based and specialized solvers while offering generality and extensibility across domains (Yang et al., 18 Jun 2025).

1. Foundations and Motivation

Combinatorial optimization presents intractable search spaces in tasks such as TSP, CVRP, MKP, JSSP, and MaxCut. Traditional heuristics require substantial expert design and lack adaptability. Previous generation hyper-heuristics, such as genetic programming, incur high computational costs, while selection hyper-heuristics rely on manually specified rules that generalize poorly. Recent LLM-based approaches (FunSearch, EoH, ReEvo, AlphaEvolve) have advanced heuristic discovery, but typically involve expensive offline search or are highly task-specific.

HeurAgenix addresses these limitations by introducing a pipeline that (i) evolves diverse heuristic strategies without external solvers, and (ii) selects among them during problem-solving via a perception-driven, adaptive mechanism, thus uniting generalizability with real-time adaptability.

2. Two-Stage Framework Structure

The HeurAgenix architecture partitions into two sequential stages:

Stage	Description	Key Components
Heuristic Evolution	Automatic generation and refinement of diverse heuristic strategies via LLM-driven mutation analysis	Seed heuristic, LLM, evolution pool
Dynamic Selection	Inference-time adaptive selection among evolved heuristics, using an LLM or fine-tuned selector	State encoder, LLM/fine-tuned model, Monte-Carlo evaluator

Let $\mathcal{Z}$ denote problem states, $\mathcal{O}$ operation space, and $\mathcal{H}$ the pool of heuristics. A trajectory $S = \{(z_i, O_i)\}_{i=0}^{n-1}$ determines solution cost $C(S)$. The selection policy is defined as $$\pi: \mathcal{Z} \times \{0, \dots, T\} \rightarrow \mathcal{H}$$, where at each step, a heuristic $H \in \mathcal{H}$ is chosen based on the current state.

3. Heuristic Evolution via Contrastive Mutation

The evolution phase iteratively refines a seed heuristic $H_{\mathrm{seed}}$ by mining contrastive solution pairs:

1. For each instance $d \in \mathcal{D}_{\mathrm{evo}}$, run $H_{\mathrm{seed}}$ to produce $\mathcal{O}$0 with cost $\mathcal{O}$1.
2. Up to $\mathcal{O}$2 perturbations: select indices $\mathcal{O}$3, replace operations $\mathcal{O}$4, roll out to $\mathcal{O}$5. If $\mathcal{O}$6, store mutation list $\mathcal{O}$7.
3. Compute per-mutation improvement $\mathcal{O}$8, identify critical operation $\mathcal{O}$9.
4. Invoke LLM to generalize the critical mutation: $\mathcal{H}$0 The LLM outputs an abstracted “evolution strategy,” e.g., “replace distance-only scoring with weighted distance and future-cost estimate.”
5. Iteratively refine $\mathcal{H}$1 by querying LLM with evolution strategies and validation performance: $\mathcal{H}$2 where $\mathcal{H}$3 is average validation cost.

Algorithm 1 in the source (Yang et al., 18 Jun 2025) formalizes this process. As illustrated (see Figure 1 in the source), multiple rounds may sequentially change the start-node rule, scoring function, insertion method, candidate filtering, and integrate local search (e.g., periodic 2-opt in TSP).

4. Adaptive Heuristic Selection and Test-Time Search

Upon constructing the heuristic pool $\mathcal{H}$4, the selector must adaptively pick the best-suited heuristic for each decision point.

The optimal policy is recursively defined (Equation 1): $\mathcal{H}$5 where $\mathcal{H}$6 consecutive applications of $\mathcal{H}$7 are considered.

The practical pipeline is hybrid: an LLM filters candidate heuristics $\mathcal{H}$8. For each $\mathcal{H}$9, Monte-Carlo evaluation estimates $S = \{(z_i, O_i)\}_{i=0}^{n-1}$0, and the minimal-expected-cost heuristic $S = \{(z_i, O_i)\}_{i=0}^{n-1}$1 is applied.

Selectors may be instantiated as large-scale LLMs (e.g., GPT-4o for zero-shot selection) or as fine-tuned lightweight models (e.g., Qwen-7B with LoRA) for lower inference cost.

5. Dual-Reward Supervised Fine-Tuning

Robust selection learning is complicated by noisy supervision. HeurAgenix leverages a dual-reward fine-tuning protocol:

5.1 Preference-Based Outcome Reward (POR):

Given noisy rollout scores $S = \{(z_i, O_i)\}_{i=0}^{n-1}$2, heuristics are ranked and thresholds $S = \{(z_i, O_i)\}_{i=0}^{n-1}$3, $S = \{(z_i, O_i)\}_{i=0}^{n-1}$4 set. POR is computed as: $S = \{(z_i, O_i)\}_{i=0}^{n-1}$5 where $S = \{(z_i, O_i)\}_{i=0}^{n-1}$6 is the ranking of $S = \{(z_i, O_i)\}_{i=0}^{n-1}$7 among candidate heuristics.

5.2 Context-Perception Reward (CPR):

To incentivize state understanding, the selector predicts features $S = \{(z_i, O_i)\}_{i=0}^{n-1}$8, receiving: $S = \{(z_i, O_i)\}_{i=0}^{n-1}$9 5.3 Combined Training (Algorithm 2):

The total reward is a weighted combination: $C(S)$0 Fine-tuning employs a group-relative policy optimization (GRPO) objective incorporating clipped loss and KL-regularization.

6. Experimental Protocol and Benchmarking

HeurAgenix is evaluated on five canonical CO problems using open benchmark suites:

Problem	Test set	Validation set
TSP	TSPLIB (12)	TSPLIB (7)
CVRP	CVRPLIB (6)	CVRPLIB (7)
MKP	OR-Library (10)	OR-Library (7)
JSSP	LA01–LA20 (20)	LA01–LA20 (10)
MaxCut	g1–g10 (10)	g1–g10 (10)

Baselines include traditional solvers (GLS, ACO, OR-Tools, QICSA, PSO, GWO, SS, CirCut, VNSPR), as well as LLM-driven hyper-heuristics (EoH+GLS, ReEvo+GLS, ReEvo+ACO). The principal metric is optimality gap: $C(S)$1 where $C(S)$2 is solution cost and $C(S)$3 is optimal. Runtimes are capped at two hours per instance, with experiments repeated three times for variance estimation. Key hyperparameters: evolution uses 2,000 LLM calls and up to $C(S)$4 perturbations per round; solving phase applies $C(S)$5 steps with 10-fold Monte Carlo; fine-tuning uses LoRA (rank 32) with Qwen-7B, AdamW (8-bit) optimizer, and GRPO batch $C(S)$6.

7. Empirical Results, Analysis, and Limitations

7.1 Evolution Outcomes

Heuristic evolution consistently reduces average optimality gaps from approximately 20–60% (seed heuristics) down to single or low digits. For example, a TSP nearest-neighbor heuristic is evolved from 25% to 9% gap through five rounds.

7.2 Selection and Solving Performance

With the GPT-4o selector, HeurAgenix demonstrably outperforms other LLM-based hyper-heuristics and matches specialized solvers across all testbeds. On TSP, it achieves an average gap of 0.50%, compared to 0.98% for the next-best method.

Fine-tuned Qwen-7B with dual-reward achieves parity with GPT-4o (0.50% average gap on TSPLIB), significantly outperforming raw Qwen-7B (5.01%) and even GRPO-only tuning (4.39%). POR+CPR ablation achieves 0.59%.

Test-time search further enhances performance: small Monte-Carlo rollout budgets ($C(S)$7) suffice to reduce gap from approximately 3% (no search) to 0.6%. The dual-reward selector outperforms all others across rollout budgets.

7.3 Observed Advantages and Limitations

HeurAgenix excels in problem diversity (routing, scheduling, partitioning) and efficient scaling via fine-tuned selectors. However, very rare LLM filtering errors can exclude critical heuristics, marginally degrading outcomes. The evolution stage’s effectiveness is tied to the diversity and informativeness of perturbation trials; extremely rugged instance landscapes may necessitate additional search.

Open directions include adaptive data-driven thresholding for POR, evaluation across a broader spectrum of LLM architectures, and generalization to arbitrary finite-state Markov decision processes.

Summary

HeurAgenix operationalizes automated heuristic discovery and adaptive deployment for combinatorial optimization via a two-stage architecture consisting of LLM-driven contrastive evolution and an adaptive selection phase leveraging both powerful LLMs and efficient fine-tuned models. Its dual-reward fine-tuning protocol yields robust heuristic selection even under noisy supervision. It provides state-of-the-art results across a range of standard benchmarks, closing the gap between LLM-based automation and domain-specific handcrafting, and marking a significant advance in model-driven CO solver design (Yang et al., 18 Jun 2025).