Meta-Black-Box Optimization with Ensemble Surrogate Modeling for Robustness-Accuracy Trade-off within SAEA

Abstract: Surrogate-assisted evolutionary algorithms (SAEAs) have been widely used for expensive black-box optimization problems. However, their reliance on rigid and manually designed components limits their flexibility and generalization across tasks. Meta-black-box optimization (MetaBBO) provides a promising paradigm for adaptively configuring algorithmic components. Nevertheless, existing MetaBBO methods usually control only a single component, and few studies have investigated the unified control of multi-component optimizers such as SAEAs. Moreover, the robustness-accuracy trade-off in surrogate modeling, which is crucial for stable early-stage exploration and accurate late-stage exploitation, has rarely been explicitly considered. To address these issues, we propose AdaE-SAEA, an adaptive ensemble surrogate-assisted evolutionary algorithm for expensive multi-objective optimization. AdaE-SAEA embeds SAEA as the low-level optimizer within the MetaBBO framework and jointly controls the infill criterion and ensemble-based surrogate modeling. Specifically, bagging and boosting are designed as surrogate modeling modules to adaptively balance robustness and accuracy across different search phases, while the meta-policy simultaneously selects the infill criterion to enable adaptive sampling decisions. The meta-policy is trained through reinforcement learning with parallel sampling and centralized training, improving both training efficiency and transferability. Experiments on synthetic and real-world problems demonstrate that AdaE-SAEA outperforms state-of-the-art baselines and MetaBBO-based methods. We further verify the effectiveness of TabPFN as the base surrogate model for ensemble learning. To the best of our knowledge, this is the first work to unify the control of surrogate modeling and infill criteria in SAEAs while explicitly addressing the robustness--accuracy trade-off.

• The paper introduces AdaE-SAEA that adaptively balances robustness and accuracy by switching between bagging and boosting surrogate strategies.
• It integrates a transformer-based TabPFN to deliver fast, scalable one-shot predictions for enhanced evolutionary candidate selection.
• The study demonstrates superior hypervolume performance and stability across benchmarks, highlighting practical benefits for expensive optimization tasks.

Meta-Black-Box Optimization with Ensemble Surrogate Modeling for Robustness–Accuracy Trade-off within SAEA

Introduction

This paper introduces AdaE-SAEA, an Adaptive Ensemble Surrogate-Assisted Evolutionary Algorithm, which advances the state of multi-objective black-box optimization by leveraging meta-black-box optimization (MetaBBO) with ensemble surrogate modeling. The central thesis is that the trade-off between robustness and predictive accuracy in surrogate models is critical for the performance and generalizability of surrogate-assisted evolutionary algorithms (SAEAs), especially in expensive optimization scenarios. AdaE-SAEA uniquely unifies adaptive control over both surrogate modeling strategies (bagging and boosting) and infill criteria selection, with the meta-policy trained via deep reinforcement learning. The framework employs a transformer-based Tabular Prior-Data Fitted Network (TabPFN) as the base surrogate model, aiming to achieve high scalability and fast inference without the prohibitive cost of Gaussian Process surrogates.

Overview of the MetaBBO and AdaE-SAEA Framework

MetaBBO Paradigm

The MetaBBO approach, as formalized in this work, utilizes a bi-level optimization framework: at the upper meta-level, a control policy (parameterized by a DQN) adaptively configures the components of the SAEA based on Exploratory Landscape Analysis (ELA) features from the current search; the lower level optimizer executes these configurations to perform black-box optimization. Meta-policy training exploits parallel environments and centralized buffer architecture to generalize across diverse optimization landscapes.

Figure 1: The general workflow of MetaBBO, detailing meta-level control and low-level optimizer interaction.

AdaE-SAEA Workflow

AdaE-SAEA extends this paradigm by embedding a joint action space comprising both surrogate modeling strategies (bagging, boosting) and infill criteria (five distinct options), enabling simultaneous and dynamic trade-offs between robustness and accuracy. The meta-policy is trained to maximize Pareto front expansion using a dense Markov Decision Process formulation, with state representation informed by a compact set of ELA features. Each optimization step engages the meta-policy to set both the surrogate model and infill criterion, then proceeds through evolutionary candidate generation, surrogate-assisted selection, and expensive evaluation.

Figure 3: The workflow of AdaE-SAEA, encompassing meta-policy decision, candidate generation, surrogate-based selection, and batch evaluation.

Ensemble Surrogate Modeling Strategies

Bagging and Boosting

AdaE-SAEA introduces ensemble modeling at the surrogate level to address the robustness–accuracy dilemma inherent in surrogate-assisted search. Bagging ensembles, through half-sampling, promote stability and reduce estimation variance—suitable for early-stage exploration. Conversely, gradient boosting ensembles reduce bias and elevate predictive accuracy, advantageous in convergence phases requiring precise selection. The adaptive switch between the two is mediated by the learned meta-policy, which exploits task-phase-dependent properties extracted from ELA.

Figure 2: Bagging and boosting surrogate modeling processes, capturing the trade-off between robustness and accuracy at different optimization stages.

TabPFN as Base Surrogate

TabPFN—an amortized-probabilistic transformer—is utilized for its superior one-shot prediction capability in tabular, small-data regimes, delivering both prediction means and uncertainty estimates. This avoids the scalability limits of GPs and renders the frequent policy/surrogate updates in MetaBBO computationally tractable.

Benchmark Evaluation and Numerical Results

Training Stability and Candidate Generation

AdaE-SAEA’s hybrid candidate generation (NSGA-III + CDM-PSL + qNEHVI) outperforms single-strategy baselines in reward accumulation per evaluation, demonstrating both improved exploration and reduced variance during meta-policy training.

Figure 4: Average reward per expensive evaluation for hybrid vs. baseline candidate generation, indicating stability and elevated reward rates.

State-of-the-Art Comparisons

AdaE-SAEA achieves superior hypervolume (HV) performance across standard multi-objective benchmarks (ZDT, DTLZ, and real-world suites), securing the best or near-best results on nearly all test problems. Notably, in most cases, AdaE-SAEA statistically outperforms both classical Bayesian optimization and MetaBBO baselines that lack unified multi-component control. The method exhibits strong generalization in leave-one-task-out cross-validation on unseen, higher-dimensional tasks.

Sensitivity and Ablation Analysis

Sensitivity tests indicate AdaE-SAEA’s robustness to moderate changes in evaluation batch size, with marginal loss only on extreme deviations from the default setting. Ablations show that both the hybrid candidate generation and the unified meta-policy-controlled surrogate/infill mechanism are critical—removing either leads to significant degradation, with surrogate modeling control being slightly more impactful.

Figure 5: Sensitivity analysis and ablation study of AdaE-SAEA using logarithmic hypervolume ratio (base 2).

Surrogate Model Evaluation

In direct comparisons, TabPFN as surrogate outperforms traditional GP in multi-objective settings, especially on small data slices. Bagging exhibits enhanced early-stage robustness, boosting offers sharper late-stage accuracy at the cost of stability, and the policy-driven adaptive combination in AdaE-SAEA achieves both goals consistently.

Figure 6: Comparison of surrogate modeling strategies, showing TabPFN-ensemble-based AdaE-SAEA achieves strong robustness and accuracy trade-off.

Theoretical and Practical Implications

AdaE-SAEA pushes the envelope in MetaBBO for SAEAs by systematically addressing the multi-component adaptability problem and recognizing the phase-dependent nature of surrogate modeling utility. The explicit meta-learned control over bagging/boosting shifts the paradigm away from fixed, manually-designed evolutionary frameworks, enabling robust and precise search across widely variable black-box landscapes. The use of TabPFN as a surrogate further establishes the practicality of transformer-based, amortized surrogate modeling for real-world, computation-constrained scenarios.

On the practical front, the approach is particularly relevant to fields such as engineering design and scientific experimentation, where expensive simulations or limited physical trial budgets demand adaptive, low-evaluation search strategies. The demonstrated transferability and stability suggest direct application to complex, high-dimensional real-world optimization.

Future Directions

The combinatorial (Cartesian product) action space in AdaE-SAEA, while powerful, scales poorly with additional algorithmic components. Hierarchical reinforcement learning or multi-agent formulations may alleviate this problem, enabling even richer adaptability. Computational efficiency for large ensembles also remains an open issue, inviting research into lightweight or sparse ensemble learning paradigms. Finally, extending the method to dynamic, constrained, or noisy multi-objective formulations could further broaden its applicability.

Conclusion

AdaE-SAEA represents a comprehensive advance in surrogate-assisted multi-objective black-box optimization, providing a unified meta-policy for joint surrogate modeling and infill criterion selection. Through adaptive control of ensemble surrogates and the integration of scalable transformer models for tabular data, AdaE-SAEA achieves robust, accurate, and generalizable optimization performance in challenging, expensive problem regimes. Its methodological contributions set the stage for future developments in highly adaptive, meta-learned black-box optimization systems.