Multi-Objective Bayesian Optimization (MOBO)

• Multi-Objective Bayesian Optimization (MOBO) is a probabilistic framework that models conflicting black-box objectives with Gaussian processes and Pareto-aware acquisitions like EHVI.
• It preserves the vectorial structure of multi-objective problems, enabling efficient exploration of the Pareto front while quantifying uncertainty.
• MOBO has demonstrated superior performance in applications such as molecular design, achieving faster convergence and greater diversity than scalarized approaches.

Multi-Objective Bayesian Optimization (MOBO) is a sample-efficient probabilistic framework for optimizing black-box vector-valued objectives subject to conflicting trade-offs. Unlike classical approaches that scalarize objectives a priori, MOBO preserves the vectorial structure, explores the Pareto front, and quantifies uncertainty via surrogate modeling—predominantly Gaussian processes (GPs)—and specialized acquisition functions such as Expected Hypervolume Improvement (EHVI). MOBO has demonstrated practical superiority over scalarization in sparse and expensive evaluation regimes, notably in molecular, engineering, and systems design, and it is supported by a growing suite of algorithmic, theoretical, and empirical advances.

1. Problem Formulation, Pareto Set, and Hypervolume

The canonical MOBO setting considers maximizing a vector-valued function $$\mathbf f(\mathbf x) = \bigl(f_1(\mathbf x), \dots, f_m(\mathbf x)\bigr)$$ over a feasible domain $\mathcal X$, where $m\ge2$ objectives are in competition (Yong et al., 18 Jul 2025). Pareto dominance is defined as: $\mathbf x \succ \mathbf x'$ iff $f_i(\mathbf x)\ge f_i(\mathbf x')$ for all $i$ and there exists at least one $j$ with $f_j(\mathbf x)>f_j(\mathbf x')$. The Pareto set $\mathcal P^* \subset \mathcal X$ consists of all non-dominated solutions; its image in $\mathbb R^m$ is the Pareto front.

The principal global optimality criterion in MOBO is the hypervolume (HV) indicator. Given a reference point $\mathbf z\in\mathbb R^m$ dominated by all Pareto points, the hypervolume of a finite front $Y=\{\mathbf y^1,\ldots,\mathbf y^k\}$ is

$$\mathrm{HV}_\mathbf z(Y) = \int_{\mathbf u\in\mathbb R^m} \mathbb I\Bigl(\exists\,\mathbf y\in Y: \mathbf y \succeq \mathbf u \text{ and } \mathbf u \succeq \mathbf z\Bigr)\, d\mathbf u,$$

corresponding to the Lebesgue measure of the region covered by $Y$ and dominating $\mathbf z$. Maximizing HV simultaneously incentivizes convergence toward the true Pareto front and trade-off diversity (Yong et al., 18 Jul 2025).

2. Pareto-Aware Acquisition: Expected Hypervolume Improvement (EHVI)

Central to MOBO's efficacy is the direct targeting of Pareto front advancement via acquisition functions that reflect vector-valued uncertainty. The prototypical Pareto-compliant acquisition is the Expected Hypervolume Improvement (EHVI) (Yong et al., 18 Jul 2025, Rodrigues et al., 2023)

$$\mathrm{EHVI}(\mathbf x) = \mathbb E\Bigl[\mathrm{HV}_\mathbf z(\mathcal P_t \cup \{\mathbf Y\}) - \mathrm{HV}_\mathbf z(\mathcal P_t)\Bigr],$$

where $\mathcal P_t$ is the current front, and $\mathbf Y = f(\mathbf x)$ is random under the GP posterior. In practice, for $m>1$, the expectation is estimated by Monte Carlo: $$\mathrm{EHVI}(\mathbf x) \approx \frac{1}{S} \sum_{s=1}^S \max \bigl\{ \mathrm{HV}_\mathbf z (\mathcal P_t \cup \{\mathbf y^{(s)}\}) - \mathrm{HV}_\mathbf z (\mathcal P_t),\,0 \bigr\},$$ where $\mathbf y^{(s)}$ are samples from the predictive Gaussian. Batch formulations such as $q$EHVI generalize this to simultaneous multi-point acquisition (Rodrigues et al., 2023). EHVI's advantage is empirical and practical: it robustly dominates fixed-weight EI scalarization in terms of faster HV growth, better front coverage, and increased diversity, especially in tight evaluation budgets and nontrivial trade-off regimes (Yong et al., 18 Jul 2025, Yong, 12 Aug 2025).

3. Surrogate Modeling and Molecular Representations

MOBO generally models each objective $f_j$ with an independent Gaussian process,

$$f_j \sim \mathcal{GP}(m_j(\mathbf x),\, k(\mathbf x, \mathbf x')),$$

with observed data $\{ (\mathbf x_i, y^{(j)}_i ) \}$ and typically zero mean. The choice of kernel is application-dependent:

• For molecular design, count-aware "MinMax" kernels on extended connectivity fingerprints (ECFPs) are effective (Yong et al., 18 Jul 2025): $$k_{\rm MinMax}(\mathbf x, \mathbf x') = \frac{\sum_{d=1}^D \min(x_d, x'_d)}{\sum_{d=1}^D \max(x_d, x'_d)}.$$
• For binary fingerprints (e.g., molecular graphs), the Tanimoto kernel is widely used (Yong, 12 Aug 2025): $$k_T(x,x') = \frac{x^\top x'}{\|x\|_1 + \|x'\|_1 - x^\top x'}.$$ GP hyperparameters are often fixed for controlled benchmarking or estimated by maximizing the marginal likelihood. The posterior predictive mean and variance at candidate $\mathbf x$ feed directly into EHVI or scalarized acquisitions.

4. Scalarization, Random Weights, and Limitations

Scalarization remains foundational in MOBO, especially when the front can be explored via multiple runs (Lin et al., 2022). The weighted-sum approach is

$$f_{\rm ws}(\mathbf x) = \sum_{i=1}^m w_i f_i(\mathbf x),$$

which reduces MOBO to SOBO for static weights and is amenable to standard EI,

$$\mathrm{EI}(\mathbf x) = (\mu_n(\mathbf x) - y^+)\, \Phi(z) + \sigma_n(\mathbf x) \varphi(z),$$

with $z = (\mu_n(\mathbf x) - y^+) / \sigma_n(\mathbf x)$, $\Phi$ and $\varphi$ the standard normal CDF/PDF. However, fixed-weight scalarization can recover at most one Pareto point per run, and even flexible variants (random/adaptive weights) require repeated acquisitions to cover the front, suffer in low-data scenarios, and may bias the search away from underexplored trade-off regions (Yong et al., 18 Jul 2025).

Random or adaptive schemes (ParEGO, Tchebycheff, PBI) enable global sample efficiency and may be linked to decomposition-based evolutionary approaches. Yet, empirical studies routinely find that Pareto-aware EHVI outperforms even strong deterministic or random scalarizations in terms of hypervolume, convergence speed, and diversity (Yong et al., 18 Jul 2025, Lin et al., 2022).

5. MOBO Algorithmic Workflow and Empirical Evaluation

The standard MOBO iterative workflow is:

1. Train GPs for each objective on all evaluated data.
2. For every candidate in the search pool, compute the acquisition function (EHVI or scalarized EI).
3. Select and evaluate the candidate maximizing the acquisition, update the data.
4. Refit the GPs and repeat.

Experimental setups, such as in (Yong et al., 18 Jul 2025), typically involve a molecular candidate pool (e.g., 10,000 sampled SMILES), optimization over hundreds of iterations, and statistical evaluation averaged across random seeds. Main performance metrics include the final hypervolume indicator (HVI), $R^2$ Chebyshev indicator for trade-off coverage, and diversity metrics such as scaffold uniqueness quantified via Tanimoto thresholds.

Key findings include:

Metric	EHVI (Pareto)	Scalarized EI	Domain/Setup
Final HVI, Fexofenadine	$0.4022\pm0.0661$	$0.3492\pm0.0190$	3-objective molecular MPO (see (Yong et al., 18 Jul 2025))
$R^2$ indicator, Fex.	$0.3728\pm0.0204$	$0.4360\pm0.0293$	Lower is better—trade-off error
Scaffold diversity (#Circles)	Higher at strict thresholds	Lower	Measured at Tanimoto $t \ge 0.60$

Effect-size tests confirm these gains are medium to large. EHVI achieves faster, more robust convergence and higher coverage of chemical and objective-space diversity.

6. Practical Implementation, Guidelines, and Method Extensions

Efficient MOBO demands scalable GP surrogates and computationally robust acquisition evaluation. Key practical tips include:

• Utilize high-performance GP frameworks for parallel kernel computations and MC sampling.
• Use $10^3$ or more MC samples per candidate to stabilize EHVI estimates.
• Fix surrogate hyperparameters in benchmarking studies; cross-validate or marginalize in production if resources permit.
• Monitor diverse metrics: hypervolume, scalarization-based indicators, and chemical diversity, to detect collapse to subspace exploration.

Promising research extensions and alternatives to standard Pareto/EHVI include:

• Random or adaptive scalarization (ParEGO) to approximate hypervolume.
• Alternative acquisitions: information-theoretic (PESMO), diversity-driven (DGEMO).
• Continuous Pareto set learning via parametric or neural architectures for real-time trade-off navigation (Lin et al., 2022, Cheng et al., 8 Nov 2025).
• Learned molecular embeddings (contrastive, deep autoencoding) to scale GP modeling to larger, raw chemical spaces (Yong et al., 18 Jul 2025).
• Batch/federated MOBO for multi-point parallel querying in laboratory or simulation environments.

7. Comparative Analysis and Broader Implications

MOBO's core advantage over scalarization is the preservation and direct exploitation of the Pareto geometry in high-dimensional trade-off landscapes, which is critical for sample efficiency and practical impact in settings where every evaluation is costly. The robust empirical advantage of EHVI-based MOBO over strong scalarized EI holds in molecular design and, by extension, to other domains with complex objective interactions (Yong et al., 18 Jul 2025, Yong, 12 Aug 2025). This supports using Pareto-aware acquisition as a robust default in early-stage expensive multi-objective optimization, particularly when evaluation budgets are constrained and trade-offs are nontrivial.

Alternative approaches (e.g., scalarization with random/adaptive weights) remain powerful for global exploration but require larger sample budgets and may not guarantee uniform front coverage. Newer approaches—continuous Pareto set modeling, parametric preference learning, and hybrid batch selection—extend the flexibility of MOBO and bridge evolutionary paradigms with probabilistically grounded search (Lin et al., 2022, Cheng et al., 8 Nov 2025).

In summary, multi-objective Bayesian optimization, particularly with Pareto-aware acquisitions such as EHVI, provides a theoretically sound and empirically validated foundation for navigating complex design trade-offs, convergence-diversity trade-offs, and uncertainty in expensive black-box objective landscapes (Yong et al., 18 Jul 2025, Yong, 12 Aug 2025, Lin et al., 2022).