---
title: Bayesian Grey-Box Optimisation
url: https://www.emergentmind.com/topics/bayesian-grey-box-optimisation
type: topic
---

# Bayesian Grey-Box Optimisation

Bayesian grey-box optimization (B-GBO) is a class of algorithms for global optimization of expensive objectives, defined by the partial exploitation of known model structure while modeling unknown or intractable components in a probabilistic (typically Gaussian process, GP-based) Bayesian optimization (BO) framework. B-GBO methods depart from the classical black-box BO paradigm by leveraging analytic, numerical, or structural information about components of the objective, constraints, or system architecture, enabling more efficient learning and improved sample efficiency in many classes of scientific and engineering problems.

## 1. Conceptual Basis and Problem Formulations

Classical BO treats the objective $f : X \to \mathbb{R}$ as an opaque mapping, with optimization conducted by constructing a surrogate model (e.g., GP) on observed pairs $\{(x, f(x))\}$. In contrast, B-GBO assumes access to partial structure within the objective:

**General Formulation:** Many B-GBO applications consider objectives that are composite or nested functions:
$$
f(x) = h\Big(g_1(x), g_2(x), \ldots, g_k(x)\Big)
$$
where $h: \mathbb{R}^k \to \mathbb{R}$ is a known (white-box) combining function and $\{g_j: X \to \mathbb{R}\}$ are black-box or partially observable component functions [2201.00272, 2306.05150, 2501.00967]. B-GBO generalizes to more complicated network or graph-structured models, where both the system topology and varying fidelity among nodes are leveraged [2502.14121].

Common instantiations:
- **Composite-/Nested-function B-GBO:** Known $h$, unknown $g_j$; each $g_j$ is independently modeled, $h$ is known and computable [2201.00272, 2105.04114].
- **Mixed White/Black-box Layers:** Some intermediate components $\phi_i$ are analytic, others are expensive/unknown, forming a chain or DAG [2306.05150].
- **Probabilistic programs:** Exploiting program structure to optimize over latent or parameter variables with embedded inference [1707.04314].
- **Spatial or domain priors:** Use of known spatial occupancy or information profiles to guide BO (e.g., sensor placement) [2309.05784].

## 2. Surrogate Modeling and Incorporation of Structural Knowledge

In B-GBO, the surrogate model reflects both known model structure and uncertainty in unknown components. This is typically achieved by multidimensional or multi-output GP priors placed over the unknown $g_j$ or nested layers [2201.00272, 2306.05150, 2105.04114]:
- **Multi-output GPs:** Model $g(x) \sim \mathcal{GP}(\mu_0(x), K_0(x, x'))$, inducing a generally non-Gaussian distribution over $f(x) = h(g(x))$ for nonlinear $h$.
- **Vectorized GPs in networks:** For graph-structured or networked systems, each node receives an independent GP prior on its mapping, with known dependency structure explicitly handled [2502.14121].
- **Grey-box surrogates in optimization frameworks:** For example, in trust-region methods, local Jacobian, Hessian, or GP-based surrogates are adaptively fit to (black-box) model components while analytic (white-box) equations are propagated without uncertainty [2509.01651].

Analytic tractability is maintained whenever $h$ is linear or affine; otherwise, posterior moments (mean, variance) of $f(x)$ must be approximated either by sampling, linearization, or novel techniques such as adaptive local linearization [2501.00967], composite EI with reparameterizations [2201.00272], or deterministic reformulation via sample average approximation [2105.04114].

## 3. Acquisition Functions, Optimization Algorithms, and Theoretical Guarantees

B-GBO methods extend classical BO acquisition criteria to exploit intermediate or structural information:
- **Composite acquisition:** Expected Improvement (EI), Probability of Improvement (PI), Knowledge Gradient (KG), and Upper Confidence Bound (UCB) are adapted to operate either on the composite $h(g(x))$ or on intermediate components, often requiring Monte Carlo or reparameterization for intractable integrals [2105.04114, 2201.00272].
- **Domain-guided acquisition:** Information-theoretic or spatial priors can modify acquisition, e.g., by introducing terms that favor exploration of regions with high expected activity/information [2309.05784].
- **Optimism-driven selection:** Auxiliary optimization can be posed to minimize upper confidence bounds of the composite, propagating GP prediction intervals through white-box layers [2306.05150].
- **Implicit constraint satisfaction:** In some frameworks, e.g., optimization for probabilistic programs, the structure of the generative model and constraints is enforced by code transformations [1707.04314].

Theoretical results establish that, under regularity assumptions on the white-box and black-box function classes (e.g., Lipschitz continuity, RKHS-boundedness), B-GBO can achieve cumulative regret bounds:
$$
R_T = O\left(\sum_{i\in\mathcal{I}_B}A_i\,\gamma_{i,T}\,\sqrt{T}\right)
$$
where the constants $A_i$ reflect the composite structure, and $\gamma_{i,T}$ is the maximum mutual information gain for each GP model [2306.05150]. Thus, B-GBO matches black-box BO up to multiplicative factors due to structural knowledge.

## 4. Representative Algorithms and Implementation Strategies

B-GBO has been developed in numerous algorithmic variants, tailored to specific problem classes:

- **COBALT**: Handles constrained composite objectives by multi-output GPs for black-box submodels, composite or expected-improvement acquisition functions (EI-CF and mWB2-CF), and chance-constraint relaxations for uncertainty in constraints, solved via sample-average approximation and high-performance NLP solvers [2105.04114].

- **BOIS**: Uses adaptive local linearization of the composite $f(x, y(x))$ to obtain analytic mean/variance approximations for the marginalized objective, supporting efficient LCB/EI acquisition with reduced computational complexity compared to Monte Carlo or high-dimensional auxiliary solves [2501.00967].

- **Trust-Region Filters with GP/Hessian Surrogates**: Exploit flexible fidelity switching (between polynomial, Taylor, GP surrogates) and Hessian-projected region updates for robust grey-box process optimization, achieving up to one order of magnitude gains in black-box efficiency [2509.01651].

- **MOBONS**: Networked multi-objective grey-box BO representation, supporting general function topology (including cycles), distributed GPs for each node, and batch Pareto-optimal Thompson sampling for scalable, multi-objective optimization [2502.14121].

- **Domain-guided Bayesian Optimization (DGBO)**: For tasks such as sensor placement, exploits domain knowledge (e.g., floorplan occupancy statistics) by integrating information-acquisition priors with classical EI, improving F1-scores and sample efficiency in sensor layout [2309.05784].

Implementation best practices include careful kernel structure selection, explicit separation (and modeling) of optimization vs. latent variables, exploitation of automatic differentiation for gradient-based acquisition optimization, use of sample-average approximation for intractable expectation computations, and adaptive learning of hyperparameters [2201.00272, 2105.04114, 2501.00967].

## 5. Applications and Empirical Results

B-GBO has demonstrated rapid convergence and improved performance in a range of domains:
- **Process and engineering system design:** Multi-unit flowsheets, optimal plant design under cost or environmental objectives efficiently exploit network structure, outperforming black-box BO on sample-efficiency metrics [2502.14121, 2501.00967].
- **Constrained simulator-based engineering problems:** COBALT achieves up to 3 orders-of-magnitude improvement in regret and feasibility-driven convergence for complex grey-box calibration problems [2105.04114].
- **Sensor network design:** DGBO demonstrates that embedding spatial occupancy priors yields improved activity-recognition F1-scores and reduces the required number of simulator calls by 39–59% relative to standard BO [2309.05784].
- **Thermal modeling and transfer learning:** Bayesian neural networks for grey-box RC thermal models attain sub-1°F RMSE in transfer scenarios with minimal retraining, outperforming DNNs/LSTMs, ARIMAX, and random forests [2009.05889].
- **Probabilistic program optimization:** Structure-exploiting BO outperforms standard BO and PMMH for marginal MAP inference, hyperparameter tuning, and model selection tasks [1707.04314].
- **Trust-region design in process engineering:** Hessian-projected, GP-augmented TRF reduces iterations and black-box evaluations 3–20× relative to classical methods on 25 benchmark and real-world cases [2509.01651].

## 6. Strengths, Limitations, and Outlook

**Strengths:**
- Sample efficiency improvements by constraining the feasible region using structural knowledge.
- Flexibility in incorporating black-box, white-box, and partially observed system components.
- Improved uncertainty quantification and acquisition targeting by leveraging intermediate system outputs and domain priors.
- Applicability to constrained, multi-objective, and networked systems [2201.00272, 2502.14121, 2306.05150, 2509.01651].

**Limitations:**
- Approximations may break down if the contribution of different structural components is strongly non-additive or exhibits unmodeled interactions.
- Analytic marginalization becomes intractable for high-dimensional, strongly nonlinear composites; approximations or additional sampling may be needed [2501.00967, 2105.04114].
- Theoretical regret guarantees assume correctly specified kernels and structural dependencies.
- Scalability to very high-dimensional internal representations or heterogeneous model classes remains under active research.

**Future directions** include:
- Transfer learning for grey-box BO across domains and system instances [2009.05889].
- Extension to settings with hybrid sensor/actuator types or adaptive structure (e.g., dynamic/online structure learning).
- Bayesian optimization over arbitrary network structures, including cycles, with parallel and batch query design [2502.14121].
- Enhanced surrogate models with warped or heteroskedastic GPs and automated structure discovery [2501.00967].

## 7. Representative Table: Comparison of Bayesian Grey-Box Optimization Paradigms

| Formulation    | Structure Leveraged            | Example Algorithms   |
|----------------|-------------------------------|---------------------|
| Composite      | Known $h$, unknown $g_j$      | COBALT, BOIS        |
| Nested/Chain   | Mixed white-/black-box layers | 2306.05150          |
| Networked      | Arbitrary graph/DAG           | MOBONS              |
| Prob. program  | Generative code transformation| BOPP                |
| Spatial prior  | Activity or occupancy maps    | DGBO                |

B-GBO thus provides a principled methodology for exploiting partial system knowledge in the global optimization of expensive functions. Through explicit structural modeling, surrogate learning, and domain-informed acquisition, B-GBO unifies and generalizes a broad spectrum of efficient optimization strategies for complex modern systems [2201.00272, 2509.01651, 2306.05150, 2501.00967, 2105.04114, 2309.05784, 2502.14121].

Source: https://www.emergentmind.com/topics/bayesian-grey-box-optimisation