---
title: Heterogeneous Online Ensembles (HEROS)
url: https://www.emergentmind.com/topics/heterogeneous-online-ensembles-heros
type: topic
---

# Heterogeneous Online Ensembles (HEROS)

Heterogeneous Online Ensembles (HEROS) denotes a resource-aware framework for stream learning in which a fixed pool of heterogeneous online models is maintained, every incoming labeled instance is used to evaluate the pool, and only a selected subset of models is trained at each step under explicit resource constraints. In the formulation introduced in 2025, HEROS targets “green online learning”: the objective is not only to preserve predictive quality under concept drift, but also to reduce update-time resource expenditure by choosing which models to train, rather than retraining the entire pool indiscriminately [2509.18962].

## 1. Definition, scope, and intended setting

HEROS is formulated for supervised data streams
\[
\mathfrak{S} = \{(x_t,y_t)\mid t\in\{0,\dots\}\},
\]
with \(x_t \in \mathcal X\) and \(y_t \in \mathcal Y\), in a standard prequential or test-then-train regime. The framework is motivated by the observation that stream ensembles are routinely used to adapt to evolving data distributions, yet established methods generally optimize predictive performance without explicitly modeling the resource cost of updating multiple ensemble members on every instance [2509.18962].

In this framework, “heterogeneous” refers primarily to diversity within the model pool through different hyperparameter choices, and in the experimental study also through different learner families. The heterogeneity is therefore predictive and computational at once: models may differ not only in accuracy but also in per-update cost \(\gamma_i\). This dual notion is central to HEROS, because the framework is designed to exploit cases where two models are similarly good predictors but differ substantially in training cost [2509.18962].

A defining feature of HEROS is that its prediction rule is not a classical vote or weighted average. The paper defines ensemble performance as
\[
\mathcal{L}(f) = \max_{f_i \in f} L(f_i),
\]
and prediction is made by the currently best-performing model,
\[
f_i = \arg\max_{f_i\in f}L(f_i), \qquad \hat y_t = f_i(x_t).
\]
Accordingly, HEROS is best understood as an online model-pool management framework with selective updating, rather than as a conventional aggregation ensemble in which all members contribute to each prediction [2509.18962].

The “green” aspect is operational rather than lifecycle-based. Resource-awareness is expressed through per-model update costs, subset selection, and empirical measurements in \( \mathrm{kWh} \). The paper does not present a full carbon-accounting methodology; instead, it treats sustainability as resource-efficient online training under explicit performance–cost trade-offs [2509.18962].

## 2. Formal framework and policy space

HEROS models training selection as a Markov decision process \( (S,A,\mathcal T,R) \). The pool is \(f=\{f_1,\dots,f_M\}\), and the state records the accumulated training resources invested in each model:
\[
S=\{ ( f_{1,r_1}, \dots f_{M,r_M} ) \mid r_i \in \mathbb{N}_{\geq 0}, i \in \{1, \dotsc, M\} \}.
\]
An action is a binary selection vector
\[
A=\{ a \mid a \in \{ 0,1 \}^M, \sum_{i=1}^M a_i \leq k \},
\]
where \(a_i=1\) means that model \(i\) is trained on the current labeled instance, and \(k\) is the maximum number of models trained per step. State transitions are componentwise:
\[
\mathcal{T}(s_t,a)_{(i)}=
\begin{cases}
f_{i,r_i} & \text{if } a_i = 0,\\
f_{i,r_i + \gamma_i} & \text{if } a_i = 1,
\end{cases}
\]
with \(\gamma_i \in [0,1]\) the per-update cost of model \(i\). The reward is
\[
R(s_t, a) = \sum_{i=1}^{M} ( L(f_i) + (1-\gamma_i) ) \cdot a_i,
\]
so the selection mechanism favors models that are both accurate and cheap to train [2509.18962].

The paper notes that one could introduce a weighted reward of the form \(L(f_i) + w(1-\gamma_i)\), but does not pursue that formulation because it introduces Pareto-optimal trade-off selection complexity. Instead, it develops a set of explicit training-selection policies.

| Policy | Selection rule | Emphasis |
|---|---|---|
| Random | Select \(k\) models uniformly at random | Exploration |
| perform-best | Choose the \(k\) models with highest \(L(f_i)\) | Immediate predictive quality |
| perform-worst | Choose the \(k\) models with lowest \(L(f_i)\) | Exploration of weak models |
| cheapest | Select the \(k\) models with smallest \(\gamma_i\) | Resource savings |
| expensive | Select the \(k\) models with largest \(\gamma_i\) | High-cost contrast baseline |
| CAND | Select \(\lfloor k/2 \rfloor\) by best performance and \(\lceil k/2 \rceil\) randomly | Exploitation–exploration mix |
| \(\zeta\)-policy | Select cheaper models whose performance is within a \((1-\zeta)\) factor of the best remaining model | Near-optimal performance at reduced cost |

This policy space makes clear that HEROS is not tied to a single heuristic. It is a general selective-update framework whose main novelty lies in turning update scheduling into a first-class object of study under resource constraints [2509.18962].

## 3. The \(\zeta\)-policy and asymptotic theory

The \(\zeta\)-policy is the central policy innovation in HEROS. For each of the \(k\) update slots, it starts from the best not-yet-chosen model
\[
j = \arg\max_{j\in J}L(f_j),
\]
then searches for a cheaper alternative \(f_l\) satisfying
\[
L(f_l) \geq (1-\zeta) L(f_j)
\]
and
\[
\gamma_l < \gamma_i.
\]
If such a model exists, the cheaper near-optimal model is chosen instead. The policy can also be combined with \(\epsilon\)-greedy exploration: with probability \(1-\epsilon\), \(\zeta\)-selection is used, and with probability \(\epsilon\), a random subset of \(k\) models is trained [2509.18962].

This construction gives the parameter \(\zeta\) a direct interpretation as performance slack. As \(\zeta \to 0\), the policy approaches perform-best; as \(\zeta \to 1\), it approaches cheapest. The paper explicitly interprets the method as “training near-optimal models at reduced costs,” rather than training the exact best models at every step [2509.18962].

The theoretical analysis assumes a stochastic model in which
\[
X_i = L(f_i), \qquad X_i \sim \mathrm{Beta}(\alpha,\beta),
\]
i.i.d. across models, while resource costs are uniformly distributed over
\[
\left\{ \frac{1}{M}, \frac{2}{M}, \dots, \frac{M}{M} \right\},
\]
and are independent of \(X_i\). The asymptotic regime takes \(M \to \infty\) first and then \(k \to \infty\). Under this model, the paper proves that for CAND the average selected-model performance converges in probability to
\[
\frac{1}{2}+\frac{1}{2}\mathbb{E}(X),
\]
while average resource consumption converges to \(\mathbb{E}(\gamma)\). For the \(\zeta\)-policy, with probability \(1-\epsilon\), the corresponding limits are
\[
\mathbb{E}(X\mid X\geq 1-\zeta)
\]
for performance and \(0\) for average resource cost [2509.18962].

The main asymptotic comparisons are then stated as four theorems. First, the \(\zeta\)-policy outperforms CAND in average selected-model performance when
\[
\frac{1}{2}(1-\mathbb{E}(X)) > \zeta,
\]
and, for \(X \sim \mathrm{Beta}(\alpha,\beta)\), this condition becomes
\[
\zeta < \frac{1}{2}\left(1- \frac{\alpha}{\alpha + \beta} \right).
\]
Second, its average resource consumption is lower than CAND’s. Third, relative to perform-best, the \(\zeta\)-policy is at most \(\zeta\) worse in average performance for large \(M\) and \(k\) with \(M>k\). Fourth, its average resource consumption is lower than that of perform-best [2509.18962].

These are asymptotic results under a simplified stochastic model rather than finite-sample regret guarantees. A plausible implication is that HEROS is theoretically strongest as a ranking-and-selection framework over a rich model pool, not as a fully adversarial online-learning algorithm. The paper’s own emphasis is therefore on average selected-model quality and average resource consumption, rather than on regret against a switching comparator [2509.18962].

## 4. Empirical evaluation and performance–resource trade-offs

The empirical study evaluates HEROS on 11 benchmark streams, including real-world datasets such as airlines, electricity, WISDM, covtype, and nomao, and synthetic streams with abrupt or gradual drift such as \( \mathrm{AGR}_a \), \( \mathrm{AGR}_g \), \( \mathrm{RBF}_f \), \( \mathrm{RBF}_m \), \( \mathrm{LED}_a \), and \( \mathrm{LED}_g \). The main experiments use a pool of \(M=50\) models and an update limit \(k=30\), under prequential evaluation with 3 random repetitions, AUROC as the main predictive metric, resource consumption measured in \( \mathrm{kWh} \), and Wilcoxon signed-rank testing with Holm adjustment at 95% confidence [2509.18962].

Two base-learner families are studied. For MLP pools, heterogeneity is induced through optimizer choice (Adam or SGD), learning rates \(\{5\cdot 10^{-1}, 5\cdot 10^{-2}, 5\cdot 10^{-3}, 5\cdot 10^{-4}, 5\cdot 10^{-5}\}\), and hidden-layer sizes \(\{2^4,2^6,2^8,2^{10}\}\). For Hoeffding Trees, heterogeneity is induced through different maximum byte sizes. Per-model costs are normalized so that
\[
\sum_{i=1}^M \gamma_i = 1.
\]
This cost normalization is essential, because the policy layer ranks update candidates jointly by observed performance and relative training expense [2509.18962].

For MLP pools, the main pattern is that perform-best ranks near the top in AUROC, cheapest ranks first in \( \mathrm{kWh} \), perform-worst is predictively weak, and CAND is neither the strongest in accuracy nor the most resource-efficient. The \(\zeta\)-policy, especially \(\zeta_{0.01,0.1}\), is among the best-performing methods while improving the accuracy–resource trade-off relative to purely performance-driven baselines. The reported mean ranks are 2.73 for perform-best and 2.64 for \(\zeta_{0.01,0.1}\) in AUROC, versus 8.73 for CAND. In \( \mathrm{kWh} \), cheapest ranks 1.09, perform-best 7.00, CAND 5.82, and the \(\zeta\)-variants range from 6.55 to 5.09 depending on \(\zeta\) and \(\epsilon\) [2509.18962].

The dataset-level examples reinforce this pattern. On electricity, perform-best attains AUROC \(0.9663\), \(\zeta_{0.01,0.1}\) attains \(0.9662\), and CAND attains \(0.9406\). On covtype, perform-best attains \(0.9904\), \(\zeta_{0.01,0.1}\) also attains \(0.9904\), and CAND attains \(0.9831\). On \( \mathrm{AGR}_g \), perform-best attains \(0.9768\), \(\zeta_{0.01,0.1}\) attains \(0.9772\), and CAND attains \(0.9725\). On \( \mathrm{RBF}_f \), perform-best attains \(0.8696\), \(\zeta_{0.01,0.1}\) attains \(0.8694\), and CAND attains \(0.7934\) [2509.18962].

The paper also compares HEROS to non-cost-aware online ensemble baselines: Adaptive Random Forest (ARF), Streaming Random Patches (SRP), and Shrub Ensembles. On AUROC mean rank, HEROS and ARF are both reported at 1.909, SRP at 2.182, and Shrubs at 4.000. The dataset-level comparisons are mixed but competitive: on airlines, HEROS attains \(0.750\) versus ARF \(0.681\) and SRP \(0.737\); on \( \mathrm{AGR}_a \), HEROS attains \(0.987\) versus ARF \(0.940\) and SRP \(0.971\); on \( \mathrm{AGR}_g \), HEROS attains \(0.977\) versus ARF \(0.921\) and SRP \(0.959\). Conversely, HEROS is not uniformly dominant: on \( \mathrm{RBF}_f \), HEROS attains \(0.866\) versus ARF \(0.935\), and on \( \mathrm{RBF}_m \), HEROS attains \(0.959\) versus ARF \(0.977\) [2509.18962].

For Hoeffding Tree pools, the predictive results remain strong, but the resource savings are less clear. The paper attributes this to the fact that policy computation can dominate total cost for lightweight base learners, especially because the HEROS policy layer is implemented in Python while some Hoeffding Tree components are executed through CapyMOA and Java. This implementation asymmetry is treated as a practical limitation of the measured green advantage, rather than as a contradiction of the selective-update idea itself [2509.18962].

## 5. Relation to adjacent online ensemble paradigms

HEROS belongs to a broader lineage of online ensemble methods, but its central mechanism differs from several established strands. In distributed online fusion, one line of work uses local learners that observe different views of the same event and transmit predictions to an ensemble learner. “Ensemble of Distributed Learners for Online Classification of Dynamic Data Streams” uses Perceptron Weighted Majority over local binary predictions, with additive mistake-driven updates and explicit extensions for delayed labels, missing labels, and asynchrony [1308.5281]. “Adaptive Ensemble Learning with Confidence Bounds” uses local contextual rule selection via Instance-based Uniform Partitioning together with an Anytime Hedge ensemble learner, and analyzes regret relative to the best local oracle [1512.07446]. HEROS differs from both by making the update subset itself the controlled object: all models are evaluated, but only some are trained, and the choice is governed by resource-aware policies rather than by confidence-weighted aggregation alone [2509.18962].

A second neighboring line focuses on online specialist formation and adaptive routing. “Cascading Randomized Weighted Majority” replaces a single global expert weighting by a cascade of stage-specific randomized weighted-majority learners, so that different experts can dominate different routed regions or error types [1403.0388]. “Shrub Ensembles for Online Classification” maintains a bounded population of small trees trained on recent windows and uses sparse simplex-constrained proximal updates to manage a memory-limited active set [2112.03723]. “Online Weighted Q-Ensembles for Reduced Hyperparameter Tuning in Reinforcement Learning” trains multiple DDPG agents with different hyperparameters and updates critic weights online from TD-error, yielding a fixed heterogeneous population with adaptive combination during learning [2209.15078]. Relative to these methods, HEROS is less concerned with contextual fusion or sparse convex aggregation than with cost-sensitive retraining of a fixed pool.

The distinction between online and offline heterogeneous ensembles is equally important. “eipy” is a batch framework for multimodal heterogeneous ensembles built around nested cross-validation and stacking [2401.09582]. “Improving the Reliability of Quantum Circuits by Evolving Heterogeneous Ensembles” studies offline-evolved heterogeneous ensembles of probabilistic quantum circuits aggregated by voting [2409.09103]. “Divide, Specialize, and Route” introduces Hellsemble, an offline routed ensemble in which base models are trained on progressively harder subsets and a router selects a single specialist at inference time [2506.20814]. These systems share HEROS’s interest in heterogeneity, specialization, and selective use of experts, but they do not address the online update-budget problem that defines HEROS [2509.18962].

## 6. Limitations, misconceptions, and open directions

Several aspects of HEROS invite clarification. First, despite the name “ensemble,” the framework introduced in 2025 does not use majority vote, weighted averaging, or stacking at prediction time. The paper’s rule is winner-takes-all over the currently best-performing model. A common misconception is therefore to read HEROS as a conventional prediction-combining ensemble; in the published formulation it is more precisely a heterogeneous online model-pool with selective retraining [2509.18962].

Second, the theory is intentionally stylized. It assumes Beta-distributed model performances, independent resource costs, and asymptotic limits \(M\to\infty\) then \(k\to\infty\). These assumptions support a clean comparison among CAND, perform-best, and the \(\zeta\)-policy, but they are not finite-sample guarantees for arbitrary drifting streams. Likewise, the framework assumes that all models are evaluated on every instance, so training cost is reduced but evaluation cost is not eliminated [2509.18962].

Third, the current formulation keeps the pool fixed after initialization. There is no dynamic expert birth–death mechanism, no explicit drift detector, and no delayed-label treatment. Concept drift is handled implicitly through continuous evaluation and selective retraining, rather than through resets or detector-triggered reconfiguration. This suggests a natural interface with neighboring ideas: contextual ensemble selection from Hedged Bandits [1512.07446], explicit specialist routing from CRWM [1403.0388], and memory-bounded expert replacement from Shrub Ensembles [2112.03723] all point toward possible extensions in which the HEROS policy layer is combined with adaptive pool maintenance.

A further practical limitation is implementation overhead. The paper reports that for lightweight Hoeffding Tree pools, policy computation itself can absorb a large share of the total resource budget. This indicates that the green advantage of selective training is most visible when base-model updates are themselves costly, as in the MLP pools. A plausible implication is that future HEROS variants may need hierarchical or approximate selection policies when candidate evaluation and policy execution become comparable to training cost.

Within the current literature, HEROS is therefore best characterized as a cost-sensitive sequential decision framework for online model updating: it makes heterogeneity operational by tying predictive quality to update-time resource expenditure. Its most distinctive contribution is the claim that near-optimal models can be preferable to exact best models when update costs differ, and that this preference can be formalized, analyzed, and validated empirically in stream learning [2509.18962].

Source: https://www.emergentmind.com/topics/heterogeneous-online-ensembles-heros