Multi-population Ensemble Genetic Programming

• Multi-population Ensemble Genetic Programming is a framework that decomposes the feature space into disjoint views, allowing specialized subpopulations to evolve via genetic programming.
• It integrates cooperative coevolution, ensemble learning, and multi-view analysis to construct additive symbolic classifiers with improved interpretability and efficiency.
• Empirical studies show that MEGP enhances search dynamics and generalization on high-dimensional, heterogeneous datasets by mitigating overfitting and redundancy.

Multi-population Ensemble Genetic Programming (MEGP) denotes a class of genetic-programming architectures in which multiple subpopulations evolve in parallel and are coupled through ensemble construction, cooperative coevolution, or both. In its most explicit recent formulation, MEGP is introduced as a classification framework that combines genetic programming, multi-population evolution, cooperative coevolution, ensemble learning, and multi-view learning for high-dimensional and heterogeneous feature spaces; a closely related formulation uses disjoint views of frozen Transformer embeddings and learns additive symbolic logit programs whose view-wise outputs are summed into a final classifier (Khorshidi et al., 16 Sep 2025, Khorshidi et al., 16 Sep 2025). Across these formulations, MEGP is not merely “an ensemble” in the bagging or averaging sense: its defining feature is the structured decomposition of the search space across populations and the assignment of fitness in a collaborative context.

1. Definitional scope and conceptual boundaries

In the classification framework that explicitly names the method “Multi-population Ensemble Genetic Programming,” MEGP is defined by five interacting components: genetic programming for symbolic model evolution, multi-population evolution for diversity and specialization, cooperative coevolution for coordination among populations, ensemble learning for prediction fusion, and multi-view learning for decomposing features into complementary subsets (Khorshidi et al., 16 Sep 2025). The central design premise is that a single GP population operating over the full feature space is often burdened by computational inefficiency, redundancy among features, overfitting, weak exploration of large search spaces, and premature convergence; MEGP addresses this by splitting the feature space into views and assigning one population to each view.

A second, highly concrete formulation appears in symbolic surrogate modeling for frozen Transformer embeddings. There, the overall pipeline is: extract fixed embeddings from a pretrained encoder, partition the embedding coordinates into disjoint views with Semantic-Preserving Feature Partitioning (SPFP), then train a cooperative multi-population GP with one population per view so that the final classifier is an additive sum of per-view symbolic programs (Khorshidi et al., 16 Sep 2025). This formulation is explicit that MEGP is not merely “an ensemble” in the bagging or averaging sense, but a cooperative coevolutionary, multi-population symbolic classifier whose final predictor is additive across views.

The literature also shows that the term has porous boundaries. Some closely related systems are best interpreted as MEGP-style rather than canonical MEGP. Ensemble GP with separate tree and forest subpopulations is explicitly described as co-evolutionary, cooperative, and compositional, but its two subpopulations have different ontological roles rather than acting as peer populations (Rodrigues et al., 2020). Domain-Informed Genetic Superposition Programming (DIGSP) is a structured multi-population ensemble GP with delayed cooperation and symbolic module transfer, but it is framed around separable physical mechanisms rather than generic multi-view classification (Khorshidi et al., 20 Sep 2025). This suggests that MEGP is most precisely understood as an architectural family centered on decomposition, specialization, and collaborative ensemble formation, rather than as a single invariant algorithm.

2. Multi-view decomposition and ensemble composition

A defining structural principle in MEGP is decomposition of the input space into views. In the multiview classification framework, the full feature set is split into conditionally independent feature subsets, also referred to as views, and the principal partitioning method used is SPFP; in the reported experiments there are 2 populations and therefore 2 views (Khorshidi et al., 16 Sep 2025). In the Transformer-surrogate formulation, SPFP is applied on the training set only, producing $V$ disjoint, information-preserving subsets of dimensions, called views, which are then fixed and reused unchanged for validation and test (Khorshidi et al., 16 Sep 2025).

The surrogate formulation gives the clearest additive-over-views equation. For classes $c=1,\dots,K$ and views $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$, population $v$ evolves symbolic programs $f_c^{(v)}$ using only the coordinates in $\mathcal{V}_v$, and the class-$c$ logit is

$$z_c(\mathbf{x}) = \sum_{v=1}^V f_c^{(v)}\!\bigl(\mathbf{x}_{\mathcal{V}_v}\bigr),$$

with probabilities obtained by

$$p_c(\mathbf{x}) = \mathrm{softmax}_c\bigl(\mathbf{z}(\mathbf{x})\bigr).$$

After model selection, scalar temperature scaling may be applied:

$$p_c^{(T)}(\mathbf{x}) = \mathrm{softmax}_c\!\bigl(\mathbf{z}(\mathbf{x})/T\bigr),$$

where $c=1,\dots,K$0 is chosen by minimizing validation negative log-likelihood (Khorshidi et al., 16 Sep 2025).

The multiview classification paper uses a different fusion point. Within each population, genes are combined into class logits and converted to isolated probabilities; ensemble prediction is then formed by linearly combining isolated probabilities from different populations:

$c=1,\dots,K$1

Thus published MEGP variants differ in the level at which fusion occurs: one sums explicit symbolic logit contributions across views, while another ensembles view-specific probability outputs with learned cross-population weights (Khorshidi et al., 16 Sep 2025, Khorshidi et al., 16 Sep 2025). A plausible implication is that “MEGP” names a decomposition-and-cooperation template rather than a single fixed aggregation equation.

3. Representation, objectives, and evolutionary mechanics

MEGP systems in this literature use symbolic expression trees or multi-gene symbolic individuals, but the details of representation and credit assignment vary. In the explicit classification framework, each individual encodes multiple genes, each gene is an expression tree, genes per individual equal 10, and the function set is $c=1,\dots,K$2; class logits are formed as

$c=1,\dots,K$3

followed by

$c=1,\dots,K$4

The weights and biases are optimized using a gradient-based method, so the method is hybrid in the precise sense of evolution for symbolic structure and gradient-based optimization for output-layer coefficients (Khorshidi et al., 16 Sep 2025).

The surrogate formulation likewise uses arithmetic expression trees with real constants. Its primitive set is $c=1,\dots,K$5 with protected division; the terminal set for population $c=1,\dots,K$6 consists of the embedding dimensions indexed by $c=1,\dots,K$7 plus ephemeral constants; constants are sampled uniformly from $c=1,\dots,K$8; maximum tree depth is 10; and initialization is Half-and-Half. The configuration table reports 30 runs, population size 30, max generations 150, stall generations 30, genes per individual 10, crossover probability $c=1,\dots,K$9, mutation probability $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$0, reproduction probability $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$1, batch size $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$2 data size / 50, epochs 1000, and learning rate 0.001 (Khorshidi et al., 16 Sep 2025).

Fitness design is central to MEGP’s cooperative character. In the multiview classification paper, isolated fitness is

$\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$3

ensemble fitness is

$\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$4

and hybrid selection is implemented procedurally through separate elite fractions $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$5 and $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$6 and a probability $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$7 that tournament selection uses ensemble rather than isolated fitness (Khorshidi et al., 16 Sep 2025). In the surrogate formulation, populations do not simply evolve independently; coevolution proceeds “by assembling teams (one candidate per view) for fitness evaluation,” and fitness is cross-entropy with parsimony, although the exact scalarization of cross-entropy and complexity is not provided in that paper (Khorshidi et al., 16 Sep 2025).

Model selection is also part of the published MEGP workflow. In the surrogate setting, 30 independent runs are performed per dataset, validation macro-$\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$8 is computed for each run, the 1-SE feasible set is formed, and the canonical surrogate is chosen by lexicographic minimization of complexity then depth, with further ties broken by fewer unique dimensions and then a deterministic hash of the serialized programs. Complexity is the total node count over all view/class programs, and depth is the maximum parse-tree depth across view/class programs (Khorshidi et al., 16 Sep 2025). This makes parsimony a downstream selection principle even when the exact online fitness combination is under-specified.

4. Architectural variants and neighboring designs

The MEGP literature includes several architectures that share the same core concerns—specialization, cooperation, and ensemble construction—while differing in how populations are defined and how information is exchanged.

Framework	Population structure	Interaction mechanism
MEGP for classification (Khorshidi et al., 16 Sep 2025)	2 populations / 2 views	isolated fitness, ensemble fitness, dual elitism, hybrid selection
Transformer-embedding surrogate MEGP (Khorshidi et al., 16 Sep 2025)	one population per SPFP view	team-based cooperative evaluation and additive logit summation
Ensemble GP (Rodrigues et al., 2020)	two subpopulations, trees and forests	forests are ensembles built from trees; add/remove/swap forest operators
DIGSP (Khorshidi et al., 20 Sep 2025)	3 populations tied to concrete, steel, and fiber mechanisms	delayed cooperation, second-level ENR fusion, AHSAM-mediated symbolic transfer
Mixed STGP+GSGP ensemble with pruning (Castelli et al., 2018)	20 populations: 10 STGP and 10 GSGP	online pruning of semantically redundant populations
2SEGP (Virgolin, 2020)	one population	multiple bootstrap-specific fitness values and partitioned survivor selection

Ensemble GP is especially informative because it shows a two-level cooperative arrangement in which one subpopulation evolves base learners and another evolves ensembles of those learners. The population is composed of two subpopulations, trees and forests, and each subpopulation has its own fitness function and genetic operators. Trees may belong to many forests, or none at all, and forest-level variation includes mutation to add a tree, mutation to remove a tree, crossover to swap trees between ensembles, and pruning of the best forest (Rodrigues et al., 2020). This is multi-population and ensemble-based, but it is not a symmetric island model.

DIGSP extends the design space further by making decomposition domain-informed rather than purely statistical. It uses 3 GP populations corresponding to concrete-related, steel/reinforcement-related, and fiber-related mechanisms; populations evolve independently in the early phase; top-performing individuals are fused through a second ENR layer; and when all populations stagnate, the Adaptive Hierarchical Symbolic Abstraction Mechanism applies one-way ANOVA, abstracts significant symbolic expressions into reusable constructs $\{\mathcal{V}_1,\dots,\mathcal{V}_V\}$9, injects those constructs into all populations, and prunes them if they harm predictive performance (Khorshidi et al., 20 Sep 2025). This suggests a broader MEGP design in which migration is replaced by statistically gated module transfer.

Other adjacent methods clarify what MEGP is not. The mixed STGP+GSGP ensemble paper evolves many populations in parallel and prunes populations whose best individuals become semantically similar, but it does not use migration, coevolutionary team fitness, or explicit ensemble-aware multiobjective selection (Castelli et al., 2018). The 2SEGP method shows that some benefits usually sought through multiple populations can be induced with one population and multiple bagging-specific fitness views rather than explicit demes (Virgolin, 2020). These neighboring architectures make clear that multi-population structure, ensemble construction, and cooperative credit assignment are separable design choices rather than a single package.

5. Empirical behavior, calibration, and interpretability

The most detailed empirical study of MEGP as a symbolic classifier over views is the Transformer-surrogate paper. Across 30 runs, mean performance is strong on four of five benchmarks: SST2G achieves $v$0, MNIST $v$1, CIFAR10 $v$2, and MSC17 $v$3, while 20NG is the hardest with $v$4. Calibration improves substantially after temperature scaling, with test expected calibration error changing from $v$5 on SST2G, $v$6 on 20NG, $v$7 on MNIST, $v$8 on CIFAR10, and $v$9 on MSC17; because calibration is a monotone logit rescaling, $f_c^{(v)}$0 and AUC are unchanged (Khorshidi et al., 16 Sep 2025).

That same study also provides a detailed picture of symbolic sparsity and reuse. The canonical models use only a small fraction of available embedding coordinates: 31/768 for SST2G, 113/768 for 20NG, 17/1024 for MNIST, 76/1024 for CIFAR10, and 34/1152 for MSC17. Frequency histograms show heavy-tailed reuse, UpSet overlap plots show nearly disjoint MNIST logits but more sharing for CIFAR10 and 20NG, and contribution-based importance is defined as

$f_c^{(v)}$1

PDP and ALE analyses indicate that top coordinates often behave as nearly monotone, antagonistic axes, with a third coordinate providing nonlinear correction (Khorshidi et al., 16 Sep 2025). These analyses are important because they show that MEGP can yield explicit, auditable classifiers rather than opaque evolved structures.

The explicit classification MEGP paper reports complementary evidence on generalization and search dynamics. Against a baseline GP model, MEGP shows win-tie-loss summaries on test data of up to 6 wins, 2 ties, 0 losses for Log-Loss, and typically 4 wins, 4 ties, 0 losses for Precision, Recall, $f_c^{(v)}$2, and AUC. MEGP also maintains significantly higher final population entropy than the baseline and generally achieves faster early fitness gains and lower best fitness values, especially on high-dimensional datasets such as GECR and HAPT (Khorshidi et al., 16 Sep 2025). One especially informative finding is that individuals that make up the best ensemble are often not top-ranked by isolated fitness within their own populations, which directly supports the cooperative-coevolution interpretation.

Evidence from domain-informed regression points in the same direction. DIGSP, using 3 populations and AHSAM-mediated symbolic transfer, reports lower training and test RMSE than a baseline multi-gene GP with one undivided population, better held-out performance in 76.7% of runs (23/30), and much smaller models, with median # Terms of 7 [6--8] versus 21 [19--23] and tree size of 41 [35--68] versus 204 177--259. This suggests that decomposition across populations can improve both parsimony and generalization when the partition aligns with meaningful substructure.

6. Terminological ambiguities, misconceptions, and limitations

A recurring source of confusion is that “MEGP” is not used uniformly across papers. In the SAG mill throughput study, MEGP stands for “multi-equation GP,” not multi-population ensemble GP. That method repeatedly extracts multiple equations tied to error-induced clusters of the training data and combines their predictions using cluster-size and distance-based weighting; it is therefore best characterized as a cluster-based multi-equation GP ensemble rather than a true multi-population GP architecture (Ghasemi et al., 2023). Any literature review that uses the acronym without qualification risks conflating distinct algorithmic traditions.

Several misconceptions are explicitly addressed in the recent MEGP papers. First, MEGP is not merely bagging or averaging: in the Transformer-surrogate formulation, the decomposition into views and the cooperative assignment of fitness to combinations of individuals are part of the definition (Khorshidi et al., 16 Sep 2025). Second, MEGP need not use migration or inter-population crossover: the explicit classification framework keeps feature subsets fixed and populations isolated in representation space, with cooperation occurring indirectly through ensemble-based fitness and selection (Khorshidi et al., 16 Sep 2025). Third, not every multi-population ensemble GP uses symmetric peer populations: Ensemble GP instead separates a population of components from a population of assemblies (Rodrigues et al., 2020).

The limitations reported in the literature are equally consistent. The multiview classification MEGP depends on the quality of feature partitioning, uses exactly two populations in its experiments, and incurs additional computational overhead because it adds multiple populations, isolated weight learning, ensemble formation, and ensemble weight optimization via COBYLA (Khorshidi et al., 16 Sep 2025). The Transformer-surrogate formulation explains decisions in embedding space rather than raw inputs, decouples SPFP and MEGP rather than jointly optimizing them, uses an intentionally minimal primitive set, and does not provide a direct baseline or ablation against a standard single-population GP or against non-symbolic surrogate alternatives within that manuscript (Khorshidi et al., 16 Sep 2025). DIGSP similarly relies on meaningful domain partitioning and on separability assumptions, while leaving some ensemble formulas and selection details under-specified (Khorshidi et al., 20 Sep 2025).

Taken together, these limitations indicate that MEGP is strongest when the feature space is high-dimensional, heterogeneous, or naturally decomposable, and when interpretable symbolic decomposition is more important than end-to-end monolithic optimization. This suggests a stable encyclopedia-level characterization: MEGP is best viewed as a family of decomposition-driven GP systems in which populations specialize on views, mechanisms, or roles, and in which collaboration is enforced at the ensemble level through additive superposition, probability fusion, or module transfer rather than through simple population parallelism alone.