Evolutionary Ensemble (EvE) Classification

Updated 4 July 2026

EvE is an evolutionary ensemble-learning method that integrates boosting and genetic programming to sequentially refine classification by reducing residual errors.
The method constructs a stack of classifiers where each stage handles part of the training data, leading to simpler, more interpretable models.
EvE demonstrates competitive accuracy with faster training and lower model complexity compared to standard evolutionary ensembles and methods like XGBoost.

Evolutionary Ensemble (EvE) is an evolutionary ensemble-learning method for classification in which boosting is used to construct a stack of genetic-programming classifiers rather than a parallel voting ensemble. In the formulation reported as BStacGP, each boosting epoch evolves a single champion on the current residual dataset, removes the records assigned to the champion’s pure bins, and appends the champion to a first-in, first-out stack for inference. The method is intended to address two linked problems in evolutionary ensemble learning—scalability to large datasets and interpretability of the ensemble—by shrinking the training set after each stage and by allowing many predictions to be made by only one or a few classifiers (Zhou et al., 2022).

1. Problem setting and architectural principle

EvE targets classification settings in which standard genetic-programming (GP) ensembles are expensive because they repeatedly evaluate models against the full training set, and in which conventional parallel ensembles are difficult to interpret. Its central design choice is to combine boosting and stacking sequentially. Each stage identifies one classifier that handles a subset of the data well, removes that handled subset from further training, and passes only the residual data to the next stage (Zhou et al., 2022).

The training data are written as

$\mathcal{D} = \langle X^t, Y^t \rangle,$

where $X^t$ is the matrix of input records and $Y^t$ is the vector of class labels. Each GP program maps an input $x_p$ to a scalar output

$\hat{y}_p = f(x_p).$

A defining aspect of the method is that $\hat{y}_p$ is not directly treated as the class label. Instead, scalar outputs are quantized into intervals on the real line, and the class distribution inside each interval is used to decide whether a prediction is sufficiently class-consistent to support removal of the corresponding records from the training set (Zhou et al., 2022).

This architecture makes EvE distinct from classical boosting, where weak learners are typically aggregated by voting or weighting. Here, boosting is used to construct an ordered stack of classifiers. Earlier classifiers are expected to handle common cases, while later classifiers specialize on progressively harder residual cases. The paper explicitly presents the method as simple, accurate, and efficient, especially relative to earlier evolutionary ensemble approaches and, in one large-scale comparison, relative to XGBoost (Zhou et al., 2022).

2. Sequential construction of the ensemble stack

The ensemble is built in boosting epochs. The process starts from

$Ensemble \gets [\empty]$

and iterates for up to Max_Boost_epoch stages. In each boosting epoch, a fresh GP population is initialized, evolved for up to Max_GP_epoch generations, and reduced to a single champion. Records correctly handled by that champion are removed from the current training set, and the champion is pushed onto the stack (Zhou et al., 2022).

Fitness evaluation uses a Gini index–based criterion,

$GiniIndex + \alpha (\%Used\_bins),$

where GiniIndex rewards useful class separation in the output bins and $\alpha$ regularizes by the fraction of bins used. The paper gives the histogram-based quantity

$hist(i, c) = \begin{cases} \frac{Count(i, c)}{Total(i)\times \#Inst(c)} & \text{if } Total(i)\neq 0 \ 0 & \text{otherwise} \end{cases}$

and states that the overall fitness is accumulated over bins and classes (Zhou et al., 2022).

Within each new population, individuals are ranked; the worst fraction (Gap) is removed to form the parent pool; and offspring are produced by cloning some parents, adding a single new node to clones, and mutating parameters. The emphasis is not on evolving one globally optimal GP tree over the full training set, but on repeatedly evolving a locally effective champion over the current residual. This training regime is the main source of the claimed reduction in computational cost (Zhou et al., 2022).

For the five small-scale UCI tasks, the reported parameterization includes Max_Boost_epoch = 1000, Max_GP_epoch = 30, NumBin = 2, purity threshold $X^t$ 0, and $X^t$ 1. Two settings are evaluated, described as fast/simple and slow/complex, differing mostly in population size and parent pool size (Zhou et al., 2022).

3. Residual dataset, purity, and champion selection

The residual dataset is the core mechanism that makes EvE boosting-like. After a GP program maps all current training records to scalar outputs $X^t$ 2, those outputs are quantized into NumBin intervals. Each interval is classified as Empty, Pure, or Ambiguous. The paper defines pure bins by a purity threshold $X^t$ 3; in words, a bin is pure if the majority class dominates strongly enough (Zhou et al., 2022).

Once a champion has been identified, all records mapped to the champion’s pure bins are removed: $X^t$ 4 where $X^t$ 5 is the set of records assigned to those pure bins. The residual dataset at stage $X^t$ 6 is therefore exactly the part of the training set not covered by the champion’s pure bins (Zhou et al., 2022).

Champion selection requires two conditions: the program must produce at least one pure bin, and its fitness must be better than the best fitness found so far. This criterion is narrower than ordinary best-individual selection, because a high-fitness tree that never yields a pure bin does not qualify as a champion. The process terminates either when Max_Boost_epoch is reached or when the residual dataset becomes empty,

$X^t$ 7

The empty-residual condition acts as early stopping once the ensemble has fully covered the training data (Zhou et al., 2022).

The residual mechanism gives the method a cascade-like structure. One model is trained, its handled region is removed, and another model is trained on what remains. The paper explicitly notes a resemblance to cascade-correlation: one model is trained, its residual is identified, and a new model is added to address the remaining error (Zhou et al., 2022).

4. Stack-based inference and interpretability

After training, EvE is deployed as a stack rather than as a parallel vote. At prediction time, the classifiers are evaluated in FIFO order: the earliest champion is queried first; if it maps the example to a pure bin, its label is used; otherwise the next champion is consulted; and the process continues until a pure bin is found or the stack is exhausted (Zhou et al., 2022).

This decision rule means that many cases are resolved by the earliest models, while rarer or harder cases fall through to deeper parts of the stack. The paper reports that, in a typical stack, the first program may classify about 57% of labels, the second 24%, the third 15%, and the fourth 4%. This allocation illustrates the intended specialization pattern of the ensemble (Zhou et al., 2022).

The interpretability claim is tied directly to this ordered evaluation semantics. Because common cases are handled by early stack elements and exceptions are handled later, the paper links the method to falling rule lists: early, relatively simple models cover the bulk of the distribution, while later models act as explicit fallback mechanisms for unresolved cases (Zhou et al., 2022).

For very large datasets, the paper proposes setting NumBin extremely high, even as high as $X^t$ 8, approximating the resolution of a floating-point number. In that regime, many bins may be pure and many may also be empty. At test time, if a queried bin is empty, the example is assigned to the closest pure bin; if the closest bin is not pure and is ambiguous, the next tree in the stack is consulted. This suggests a deployment mode in which quantization resolution itself becomes part of the coverage strategy (Zhou et al., 2022).

5. Experimental evaluation and comparative results

The empirical study reported for EvE/BStacGP consists of two benchmark settings. The first uses five UCI binary classification tasks—BCW, HEART, IONO, PARKS, and SONAR—with 70/30 stratified train/test splits. The second uses the CTU-13 intrusion detection dataset, specifically the union of normal and botnet traffic, totaling about 801,132 records (Zhou et al., 2022).

On the UCI tasks, the proposed method is compared against four evolutionary baselines from prior work: 2SEGP, eGPw, DNGP, and M3GP. The reported conclusions are that BStacGP is competitive with the best prior evolutionary ensemble methods, that Friedman ranking did not reject the null hypothesis that all algorithms are ranked equally, and that BStacGP tends to produce much simpler solutions and much faster training than 2SEGP. The paper attributes these properties to residual-focused training, which reduces both computational cost and model complexity (Zhou et al., 2022).

On CTU-13, the comparison includes BStacGP, 2SEGP, C4.5, and XGBoost. Under the main comparison, XGBoost achieves the best test accuracy, but BStacGP remains close: BStacGP test accuracy is reported as 0.953, XGBoost test accuracy as 0.9681, and 2SEGP test accuracy as 0.8419. In terms of efficiency, BStacGP is reported as about twice as fast as XGBoost and about three orders of magnitude faster than 2SEGP; in terms of model complexity, it produces models that are 2–3 orders of magnitude simpler than the comparators (Zhou et al., 2022).

A second CTU experiment constrains competitor models to similarly low-complexity settings. There, BStacGP is reported to maintain strong accuracy while the decision tree and XGBoost lose substantial performance. The paper presents this as evidence for a strong accuracy-efficiency tradeoff rather than for absolute accuracy dominance (Zhou et al., 2022).

These findings are consistent with the method’s stated objective. EvE is not presented as uniformly superior in raw predictive performance; rather, its reported contribution lies in combining competitive accuracy with lower training cost, smaller solutions, and a more inspectable prediction pipeline (Zhou et al., 2022).

Within the evolutionary ensemble literature, EvE/BStacGP occupies a specific position. Relative to bagging-based GP ensembles such as 2SEGP, it removes correctly handled records rather than repeatedly training on bootstrap resamples. Relative to coevolutionary or island-based methods, it uses a single sequential stack with each stage focused on the current residual. Relative to M3GP, it is explicitly an ensemble-construction method rather than a feature-mapping or feature-engineering method (Zhou et al., 2022).

The term “Evolutionary Ensemble” is not used uniformly across the literature. In “Evolutionary bagging for ensemble learning,” the evolving objects are the contents of the bags themselves, and the paper explicitly describes EvoBagging as its version of an “Evolutionary Ensemble (EvE)” (Ngo et al., 2022). In “Evolutionary Ensemble of Agents,” EvE denotes a decentralized framework with two co-evolving populations—functional code solvers and agent guidance states—evaluated through a synchronous race and updated by empirical Elo ratings (Yu et al., 9 May 2026). By contrast, “An Experimental Study of Adaptive Control for Evolutionary Algorithms” uses EvE in a different sense altogether: the exploration-versus-exploitation balance in evolutionary search (Tollo et al., 2014).

A common misconception is therefore to treat “EvE” as the name of a single canonical algorithm. The record supplied by these papers indicates otherwise. This suggests that the label functions more as a family name for evolutionary selection or co-evolution schemes than as a uniquely defined architecture. In the classification context of BStacGP, however, the term refers specifically to a boosting-driven stacked GP ensemble organized around champion selection, residual-dataset reduction, and FIFO inference (Zhou et al., 2022).