Hyper-Heuristic Framework & MEHH Overview

• Hyper-Heuristic Framework is a generalized methodology that automates the discovery, selection, and generation of heuristics to solve combinatorial optimization problems.
• The MEHH framework integrates MAP-Elites with genetic programming to evolve diverse, interpretable scheduling rules through systematic exploration and rigorous evaluation.
• Empirical results on RCPSP benchmarks demonstrate improved solution quality and enhanced heuristic diversity compared to traditional GP and manually designed rules.

A hyper-heuristic framework is a generalized search methodology that automates the discovery, selection, or generation of heuristics to solve classes of complex combinatorial optimization problems. In contrast to metaheuristics that focus on manipulating problem solutions directly, hyper-heuristics raise the level of abstraction by targeting the space of heuristics themselves, seeking robust, domain-independent approaches. This article presents a detailed account of the MAP-Elites based Hyper-Heuristic (MEHH) framework as applied to automated discovery of priority rules for the Resource-Constrained Project Scheduling Problem (RCPSP), as well as situating the framework conceptually and methodologically within the broader landscape of heuristic discovery and combinatorial search (Chand et al., 2022).

1. Foundations of Hyper-Heuristic Frameworks

Hyper-heuristic frameworks are characterized by a layered architecture: a high-level strategy (hyper-heuristic controller) operates over a space of low-level heuristics or heuristic-building blocks. The aim is to produce algorithms that generalize across multiple problem instances or classes, leveraging search or learning mechanisms at the heuristic level (e.g., genetic programming, reinforcement learning, or quality-diversity search).

The MEHH framework exemplifies this approach by wrapping a quality-diversity search algorithm (MAP-Elites) around the genetic programming-based discovery of scheduling heuristics (i.e., priority rules for job selection during project scheduling). This paradigm shifts focus from directly manipulating RCPSP schedules to evolving interpretable scheduling rules, searching simultaneously for high performance and structural diversity in the heuristic population.

2. MEHH: Overall Architecture and Search Workflow

The MEHH framework couples a MAP-Elites quality-diversity loop with a programmatic representation of candidate heuristics. The main stages are:

• Initialization: A population of $\lambda$ candidate priority rules, represented as genetic programming (GP) trees, is initialized. Each rule encodes an arithmetic expression that maps activity features to priority values.
• Evaluation: Each rule is evaluated on a fixed training set $T$, where it is used to generate RCPSP schedules via a Parallel Schedule Generation Scheme (PSGS). Fitness is computed as the average percentage makespan deviation from a lower bound, $$p(y) = \frac{1}{|T|}\sum_{i\in T}\frac{ms_i - lb_i}{lb_i}\times 100\%$$.
• Archive Mapping: Each rule $y$ is characterized by a feature vector $f(y)$ and assigned to a cell in a 3-dimensional MAP-Elites grid. The features are:
  1. $f_1(y)$: total number of nodes in the GP tree (structural complexity)
  2. $f_2(y)$: count of resource-related feature nodes (resource-awareness)
  3. $f_3(y)$: normalized average total schedule slack induced by $y$ (schedule flexibility)

• Variation and Selection: New candidate rules are generated by crossover (subtree exchange) or mutation (subtree replacement) applied to elites from the archive. Offspring inherit their parent’s feature-cell assignment.

• Archive Updating: If a new candidate $y$ mapped to cell $f(y)$ outperforms the existing elite or if the cell was empty, it replaces the occupant.
• Validation and Selection: After $G$ generations, all elites are re-evaluated on a separate validation set; the best elite is finally tested on a large hold-out set.

The workflow aligns with the following high-level pseudocode:

Input: training set T, generations G, batch λ, grid sizes b1,b2,b3 Initialize empty archive P: Z^(b1×b2×b3) ← Φ for gen=0 to G do for k=1 to λ do if gen==0 y ← random_tree(max_height=5) else Select parent cell f_p among nonempty cells y_p ← P(f_p) y ← variation(y_p) # crossover or mutation end evaluate p(y) on T compute feature vector f(y)=(f1,f2,f3) discretize f(y)→cell c if P(c)==Φ or p(y)<p(P(c)) then P(c)←y end end end Re-evaluate all y∈P on validation set V; select y* = argmin_y p_V(y) Evaluate y* on hold-out test set S; return y*.

3. Candidate Rule Representation and Phenotypic Feature Mapping

Candidate heuristics in MEHH are encoded as arithmetic expression trees with:

• Operators: Add, Subtract, Multiply, Divide (protected, i.e., $a/b\equiv0$ if $b=0$), Max, Min, and unary Negate
• Leaf Nodes (Features): Ten normalized activity attributes, including earliest/latest start/finish, total number of predecessors/successors, resource requirements, and derived statistics on resource needs

At scheduling time, each candidate rule computes a priority score for each eligible activity. The PSGS constructs a feasible schedule by repeatedly selecting the activity with the minimum rule value.

Phenotypically, each candidate is projected into a quality-diversity archive by:

Feature	Description	Range/Typical Value
$f_1$	GP tree size (nodes)	$[4,127]$
$f_2$	Resource-related leaf count in tree	$[0,30]$
$f_3$	Avg. normalized slack across training schedules	Empirically $[1.65,2.0]$

These features are discretized into $b_1 \times b_2 \times b_3$ grid cells, producing a structured, diversity-preserving map in which each cell contains the best (lowest fitness) rule observed within its feature region.

4. MAP-Elites Integration: Quality-Diversity Search

MAP-Elites, a prominent method in the quality-diversity family, drives the exploration–exploitation balance in MEHH:

• Initialization: Grid cells are initially unpopulated; random rules are evaluated and fill their mapped cell if vacant.
• Variation Operators: At each generation, new rules are generated with probability $p_c=0.8$ by subtree crossover and $p_m=0.2$ by subtree mutation.
• Archive Policy: For each new rule, if its target cell is unoccupied or the new candidate achieves a better fitness, that rule replaces the previous elite in the cell.

The design ensures a continuous influx of structurally and functionally diverse rules, combating the premature loss of population diversity observed in standard GP-based hyper-heuristics.

5. Empirical Results and Comparative Performance

MEHH was benchmarked against classical genetic programming hyper-heuristics (GPHH) and manually-designed priority rules across several large-scale RCPSP benchmarks:

• On 480 J30 (size-30) instances for training, and 300-size RG300 instances for validation/testing, MEHH consistently outperformed both GPHH and human-crafted rules.
• Coverage: MEHH maintained 1,900 unique individuals at termination for 8,000 archive cells, compared to approximately 37 for GPHH.
• Diversity: Grid coverage ranged from 23% (8,000-cell archive) to 66% (125-cell archive), far exceeding conventional methods.
• Solution Quality: For the largest archive ($b_1=b_2=b_3=20$, 8,000 cells), the mean makespan deviation was $1003.40\%$ (standard deviation $0.80$) versus GPHH’s $1005.93\%$ (standard deviation $2.01$), representing a statistically significant $0.25\%$ improvement.
• On hard RG300 instances, MEHH’s best rules surpassed even the strongest expert rule (e.g., $1001.5\%$ for MEHH versus $1013.7\%$ for the leading manual rule).

These results establish that the integration of quality-diversity search with heuristic evolution yields robust gains in both performance and population diversity, particularly on under-studied large problem instances (Chand et al., 2022).

6. Generalization Across Domains and Practical Implications

The MEHH framework's applicability extends beyond RCPSP. Any scheduling or combinatorial optimization task admitting constructive heuristics structured as expression trees over instance features is a candidate for this architecture. For example, in Vehicle Routing or Flexible Job Shop Scheduling, one can:

• Define relevant local features (e.g., distance to next node, remaining capacity)
• Adopt a constructive schedule-building strategy (serial or parallel)
• Specify appropriate grid features (e.g., heuristic complexity, resource sensitivity, slack)

The selection of 2–4 informative features and discretization into a multidimensional archive promotes both repertoire diversity and solution quality. This approach enables systematic exploration of the heuristic design space in domains where manual rule creation or search-based auto-generation previously suffered from convergence to a small set of similar heuristics.

7. Significance and Future Directions

The integration of MAP-Elites into hyper-heuristic frameworks represents a significant advance in automated heuristic discovery. By enforcing explicit, structured diversity through feature-based archival and maintaining a dynamic population of "elite" heuristics per region, MEHH fosters the emergence of novel, high-performing, and generalizable rules. Empirical results support claims of both improved average-case performance and substantially greater coverage of the heuristic space compared to monolithic GP or expert-rule approaches.

Future work may include:

• Extension of feature characterizations to higher-dimensional, domain-specific attributes
• Integration with self-adaptive or meta-learning controllers for archive navigation and grid parameterization
• Application to online heuristic selection in dynamic scheduling environments

This paradigm supports a robust, scalable route to automated heuristic design and provides a foundation for next-generation hyper-heuristic research, which is increasingly important as combinatorial problems grow in scale and complexity (Chand et al., 2022).