mlr3mbo: Bayesian Optimization in R

Abstract: We present mlr3mbo, a comprehensive and modular toolbox for Bayesian optimization in R. mlr3mbo supports single- and multi-objective optimization, multi-point proposals, batch and asynchronous parallelization, input and output transformations, and robust error handling. While it can be used for many standard Bayesian optimization variants in applied settings, researchers can also construct custom BO algorithms from its flexible building blocks. In addition to an introduction to the software, its design principles, and its building blocks, the paper presents two extensive empirical evaluations of the software on the surrogate-based benchmark suite YAHPO Gym. To identify robust default configurations for both numeric and mixed-hierarchical optimization regimes, and to gain further insights into the respective impacts of individual settings, we run a coordinate descent search over the mlr3mbo configuration space and analyze its results. Furthermore, we demonstrate that mlr3mbo achieves state-of-the-art performance by benchmarking it against a wide range of optimizers, including HEBO, SMAC3, Ax, and Optuna.

• The paper introduces a modular framework that enables custom Bayesian optimization pipelines in R, integrating robust surrogate models and acquisition functions.
• It employs advanced techniques like log transformation, multi-point proposals, and asynchronous optimization to enhance performance in numeric and mixed parameter spaces.
• Empirical evaluations demonstrate superior anytime performance and practical advantages over existing frameworks in hyperparameter optimization scenarios.

mlr3mbo: A Modular Framework for Bayesian Optimization in R

Introduction

mlr3mbo introduces a comprehensive and modular toolbox for Bayesian optimization (BO) within the R programming environment, designed to target both the needs of advanced research and applied HPO scenarios. The package supports a broad spectrum of BO regimes, spanning single- and multi-objective optimization, multi-point proposal strategies, advanced parallelism, input/output transformations, and robust error handling. Its primary design philosophy emphasizes extensibility—providing highly granular compositional building blocks and a transparent API that allow researchers to instantiate custom BO pipelines and practitioners to execute robust black-box optimization out of the box.

Core Principles and Algorithmic Design

Bayesian Optimization Workflow

BO is conceptualized as a sequential global optimization procedure leveraging probabilistic surrogate models (SM, typically GPs or random forests) as computationally cheap approximations of expensive black-box objective functions. The typical workflow consists of:

1. Initial design—space-filling sampling (random, LHS, Sobol).
2. Surrogate modeling—choice of GP or non-Gaussian models as dictated by the structure of the search space (purely numeric vs. mixed/hierarchical).
3. Acquisition function definition—e.g., EI or LCB, balancing exploitation and exploration.
4. Acquisition function optimization—global (CMA-ES, DIRECT) or local (LS) optimization strategies potentially subject to restarts or multi-modalities.
5. Evaluation, archival, and sequential update, subject to a user-defined or composite termination criterion.

mlr3mbo exposes each element as a pluggable component, tightly integrated via formal object-oriented patterns, enabling direct extension or replacement of every element in the optimization process.

Example: Black-Box Regression in R

Figure 1: Sinusoidal function used as an example.

The package provides convenient programmatic abstractions to define objectives, parameter domains/codomains, surrogates, and entire optimization setups. As illustrated in (Figure 1), a sinusoidal 1D function is defined, and relevant abstractions such as the Objective, ParameterSet, and Oi (optimization instance) are constructed with minimal overhead, facilitating reproducible experimental workflows.

Surrogate Models and Acquisition Function Interplay

A critical design decision in BO pertains to surrogate and acquisition function compatibility. mlr3mbo's interface supports any regression learner from the mlr3 core ecosystem; for BO use, only those supporting both posterior mean and standard deviation (e.g., via predict_type = "se") are usable, imperative for credible uncertainty estimation. GPs (via DiceKriging) are the default for purely numeric spaces; random forests with flexible uncertainty estimators (ESD, LTV) are preferred for mixed or hierarchical parameterizations.

The acquisition function layer is fully modular—for example, EI and LCB are both implemented, and alternatives for asynchronous or multi-objective BO (e.g., stochastic variants, hypervolume-based) are accessible with one-liner modifications.

Figure 2: Expected Improvement (solid dark gray line) based on the mean (dashed gray line) and uncertainty (gray shaded area) prediction of the GP SM trained on an initial design of four points (black).

(Figure 2) visualizes the expected improvement landscape overlayed with predictive uncertainty, showcasing local variance quantification crucial for balancing exploration vs. exploitation.

Advanced Features

Transformations, Randomization, and Robustness

Both input and target transformations (scaling, log/Box-Cox/Yeo-Johnson, etc.) are natively supported to stabilize regression targets (standardizing heavy tails or mitigating heteroscedasticity) and address non-stationarity in feature spaces. The package can automatically invert transformations for acquisition computation unless acquisition logic is tailored for the transformed scale (e.g., LogEI). Random interleaving and warmstarting mechanisms are included, crucial for high-dimensional settings or batch transfer learning workflows.

Robust error handling is a pervasive feature—numerical errors in surrogate training (e.g., GP fitting singularities) or acquisition failures are trapped and recovered via fallbacks, ensuring continuity of the optimization process.

Mixed and Hierarchical Spaces

mlr3mbo's native support for mixed and conditional hierarchies extends GP-based approaches by leveraging state-of-the-art forest models with advanced uncertainty estimation. The package natively encodes missing/deactivated variables, automatically reconciling search space topology during mutation in LS or neighbor generation.

For acquisition function optimization, stochastic or local strategies are implemented for non-purely-continuous spaces, maintaining efficiency and efficacy in combinatorial/structurally constrained domains.

Parallel and Asynchronous Bayesian Optimization

The framework supports multi-point batch proposals (via constant liar and q-EI) and fully asynchronous decentralized MBO—an architectural advance for HPC/cluster regimes, where centralized proposal is a bottleneck. Workers update independently on all available information, simulate in-progress results (constant liar), and diversify via stochastic acquisition hyperparameterization. Communication is orchestrated via a lightweight database backend, with all core logic abstracted for extensibility.

Hyperparameter Optimization and AutoML

mlr3mbo deeply integrates with the mlr3tuning and mlr3pipelines ecosystems, supporting CASH and full AutoML workflows. It provides seamless orchestration of complex ML pipelines: traversing algorithm selection, preprocessing, hyperparameter tuning, and even nested resampling for unbiased model assessment in a unified interface. Graph-based abstractions allow arbitrary dependency structures, and the search space formalism supports conditional hyperparameters and type-aware constraints, crucial for AutoML.

Empirical Evaluation and Numerical Results

A thorough empirical evaluation is conducted using the YAHPO Gym surrogate-based HPO benchmark suite, covering both fully numeric and mixed/hierarchical regimes. Configurations are meta-optimized via coordinate descent, and the "random search normalized score" (RSNS) is analyzed, providing direct interpretability relative to mundane baselines under constrained budgets.

Key findings:

• On numeric benchmarks, the optimal default configuration uses a GP with log-transformed output, LCB acquisition ($\lambda=3$), and CMA-ES for acquisition optimization, achieving a mean RSNS of 1.19 and strong anytime performance.
• On mixed/hierarchical problems, a large random forest with LTV, log output, and LS yields a mean RSNS of 0.82. Ablation studies demonstrate substantial performance degradation with small forest size, absence of log transformation, or alternative acquisition optimizers.
• Against state-of-the-art competitors (Ax, SMAC3, HEBO, Optuna), mlr3mbo matches or surpasses SOTA anytime and final performance for both numeric and complex spaces. mlr3mbo's runtime overhead is consistently low, except in the highest-dimensional mixed regimes, where acquisition function optimization dominates overhead (but does not yield improved performance).
• Significant empirical claims: strong acquisition function optimizers (CMA-ES and LS) and log output transformation are crucial for optimal performance; weak surrogate variance estimation or small forests are highly detrimental in mixed/hierarchical cases.

Implications and Outlook

mlr3mbo's design for maximal composability has immediate implications for both method development and scalable, production-grade HPO. The clear separation of surrogates, acquisition logic, and optimizers simplifies experimentation with new modeling paradigms, custom uncertainty estimation, or novel acquisition strategies. The robust, parallel, and asynchronous execution model, along with the resilient error handling and integration with the R ML stack, makes it appropriate for high-throughput scientific discovery, automated engineering, and AutoML research.

Key practical takeaways:

• Carefully tuned defaults, particularly for surrogate/acquisition optimizer combinations and output transformation, are essential—oversimplified configurations (small random forests, naive acquisition optimization) can halve performance.
• Seamless interoperability with the mlr3tuning/AutoML toolchain and support for complex conditional spaces makes mlr3mbo a pragmatic choice for real and academic HPO/AutoML use cases.
• Surrogate-based benchmarks accelerate extensible meta-optimization research, but empirical validation on real, noisy target functions remains an open area.

Theoretical implications include modular experimentation with BO components, deep integration in AutoML meta-systems, and support for distributed, decentralized, or even federated optimization regimes. Future directions include benchmarking on non-surrogate, noisy HPO problems, continued development of kernelized/mixed surrogates, tighter optimization of acquisition function search in high-dimensional spaces, and extensions to scalable neural surrogates and advanced acquisition heuristics.

Conclusion

mlr3mbo establishes a new standard for modular, extensible Bayesian optimization in R, matching or outperforming competitive frameworks in accuracy and efficiency, while enabling method development and rigorous empirical evaluation. Its composable building blocks, robust handling of search space complexity, and integrated parallel/asynchronous execution position it as a powerful tool for both research and applied AutoML/black-box optimization contexts.

See (2603.29730) for the original paper and full technical details.