---
title: 'Grey-Box Optimisation: Methods & Applications'
url: https://www.emergentmind.com/topics/grey-box-optimisation
type: topic
---

# Grey-Box Optimisation: Methods & Applications

Grey-box optimisation encompasses computational frameworks wherein the objective function and/or constraints consist of both analytically tractable (glass-box) components and black-box models lacking explicit derivatives. These scenarios arise pervasively in engineering, process systems, simulation-based design, and combinatorial optimisation, reflecting realistic solver settings where partial structural, algebraic, and gradient information is available but key parts remain opaque. By systematically exploiting known model structure—either via analytic derivatives, problem decompositions, or mechanistic knowledge—grey-box optimisation algorithms attain marked improvements in sample efficiency, convergence guarantees, and scalability relative to black-box approaches, while retaining generality and robustness against model uncertainty.

## 1. Problem Formulation and Modelling Structure

The canonical grey-box problem is:

\[
\min_{x\in\mathcal X}\;f(x) \;=\;\min_{x\in\mathcal X}\;[f_g(x) + f_b(x)] \quad \text{s.t.}\quad c(x)=0
\]

where:

- \(f_g(x)\) represents the glass-box (analytic, differentiable) component,
- \(f_b(x)\) captures the black-box (non-analytic, simulation or data-driven) part, approximated via local surrogates,
- \(c(x)\) can mix glass-box and black-box algebraic or simulation-based constraints.

Explicit decoupling via variable partitioning and auxiliary outputs is standard, e.g., splitting \(x = [w, y, z]\) with \(y = t(w)\) representing simulator responses or black-box outputs, and \(z\) denoting purely glass-box variables [2511.18998]. In combinatorial and pseudo-Boolean settings, the objective often decomposes as a sum over localized subfunctions:

\[
f(\mathbf{x}) = \sum_{i=1}^{m} f_{i}(\mathbf{x}_{C_i})
\]

with each \(f_{i}\) acting only on a small mask \(C_i\) of variables, and the variable interaction graph (VIG) encoding dependency structure [2504.11864, 1707.03093]. In Bayesian optimisation, grey-box models arise when the system performance \(f(x)\) is expressible as compositions of known and unknown mappings, e.g., \(f(x, y(x))\) where \(y\) is GP-modelled [2501.00967], or nested multi-level chains [2306.05150].

## 2. Algorithmic Approaches: Trust-Region and Surrogate Modelling

Deterministic grey-box optimisation in continuous domains typically exploits trust-region-based strategies [2511.18998, 2509.01651, 1912.12637]. At each iterate, glass-box components are treated with exact or linearised differentials, while black-box portions are locally approximated by surrogates. Standard surrogate forms include:

- Polynomial interpolation (linear, quadratic)
- Taylor expansions (if gradients available)
- Gaussian-process regression (GP)
- Hybrid combinations (e.g., Taylor + GP-residual)

A typical trust-region subproblem is:

\[
\min_{d}\; f_g(x_k + d) + m_b^k(d) \quad \text{s.t.}\;\|d\|\le\Delta_k,\; h(x_k+d)=0,\; g(x_k+d)\le0
\]

with \(m_b^k(d)\) constructed to satisfy fully-linear or fully-quadratic error bounds in the trust-region [2511.18998].

Globalisation and step acceptance are performed via filter methods—infeasibility/objective pairs—or funnel mechanisms tracking a single scalar error bound (surrogate mismatch or constraint infeasibility), as in the trust-region funnel algorithm [2511.18998]. New variants incorporate second-order information (local Hessian projections) in trust-region geometries, yielding dramatic reductions in black-box calls and enhanced robustness against tuning parameter choices [2509.01651]. Restoration phases and surrogate management (geometry repair/poisedness maintenance) further refine trajectory selection and ensure convergence [1912.12637].

## 3. Bayesian Grey-Box Optimisation: Composite and Nested Settings

Bayesian frameworks extend grey-box optimisation to situations where expensive-to-evaluate constituents are modelled with Gaussian processes, and compositional or multi-fidelity structure is leveraged [2201.00272, 2306.05150, 2501.00967, 2105.04114]. Key domains include:

- Composite objectives: \(f(x)=g(h(x))\), with known link \(g\) and black-box vector-valued \(h(x)\);
- Multi-fidelity settings: querying cheaper approximations \(h(x,z)\) at varying fidelity parameter \(z\);
- Nested and multi-level models: cascading chains \(f(x)=e^T\Phi_{m-1}(\Phi_{m-2}(\dots\Phi_0(x)))\) encompassing white- and black-box stages.

GP surrogates are maintained separately for each unknown function, propagating uncertainty through known mappings to compute posterior moments, expected improvement, or lower/upper confidence bounds [2501.00967]. Notably, the BOIS algorithm achieves analytical forms for acquisition functions by adaptive linearisation around GP means, avoiding expensive sampling or high-dimensional auxiliary optimisation [2501.00967]. Optimism-driven UCB strategies (solving a constrained auxiliary problem over plausible GP confidence intervals for all black-box components) attain regret bounds matching black-box BO up to a constant, but frequently improve empirical sample efficiency by orders of magnitude [2306.05150, 2105.04114]. Derivative information in glass-box parts is exploited analytically for faster acquisition maximisation and tighter uncertainty quantification [2105.04114].

## 4. Discrete, Combinatorial, and Evolutionary Grey-Box Methods

In discrete optimisation, grey-box techniques enhance local search and evolutionary algorithms by exploiting known subfunction structure and variable interdependencies [2504.11864, 1707.03093]. Mechanisms include:

- Partitioning variable interaction graphs to form local masks for complex, nontrivial moves (including tunnelling across local optima at high Hamming distance);
- Mask-informed perturbation and crossover (e.g. Partition Crossover in scale-free MaxSat instances);
- Auxiliary metrics such as clause-satisfiability (multi-sat statistics) to guide long “wormhole” hops between disconnected regions near critical phase transitions [2504.11864];
- Factorised Distribution Algorithms (FDA): Probabilistic graphical models with variable factorisation directly inherited from the problem structure, enabling exact or message-passing sampling on junction trees [1707.03093];
- Hybrid grey-box–EDA approaches, moderating building-block disruption and recombination via interplay with graphical/structural models.

These schemes attain polynomial complexity on low tree-width instances and mitigate bottlenecks in searching hard combinatorial landscapes.

## 5. Practical Implementations and Performance Benchmarks

Open-source frameworks and solvers have codified grey-box optimisation algorithms for scalable application:

- Pyomo-based trust-region funnel method: modular surrogates, criticality/compatibility checks via small LP/QP, analytic derivatives via autodiff, restoration and parameter experimentation [2511.18998];
- DEFT-FUNNEL (Matlab): globalisation by clustering-based multistart (MLSL), self-correcting surrogate geometry, slack variable embedding, and practical input handling for white-box derivatives [1912.12637];
- TRF-Solver: Hessian-enhanced trust-region filter algorithm with surrogate switching, implemented in Pyomo with IPOPT [2509.01651];
- Bayesian grey-box optimisation algorithms (BOIS, COBALT): analytical and SAA-based acquisition optimisation, automated derivative exploitation, and explicit chance-constraint handling [2501.00967, 2105.04114].

Benchmark studies indicate marked efficiency gains in iteration counts, black-box evaluations, and CPU time compared to classical or black-box solvers; for instance, order-of-magnitude reductions in black-box calls, and robust convergence to KKT points in large-scale process flowsheet optimisation [2511.18998, 2509.01651, 1912.12637]. In combinatorial settings, multi-satisfiability-guided search recovers global connectivity and 100% success on hard MaxSat instances beyond the capability of prior state-of-the-art optimisers [2504.11864].

| Algorithm/Method      | Domain        | Efficiency Result                                       |
|----------------------|--------------|--------------------------------------------------------|
| TR funnel (Pyomo)    | Process NLPs | 2-10x fewer black-box calls vs. filter; strong globalisation [2511.18998] |
| Hessian-enhanced TRF | Process NLPs | 5-10x fewer evals; 100% success rate in benchmarks [2509.01651] |
| BOIS                 | Bayesian BO  | Equivalent/better than OP-BO, with 3x less acquisition time [2501.00967] |
| COBALT               | Bayesian BO  | 1-3 orders faster regret reduction; 50% fewer constraint violations [2105.04114] |
| MOCSM (multi-sat)    | Max3Sat      | 100% success in hard instances (<2 min); scales to 350 variables [2504.11864] |
| DEFT-FUNNEL          | Global DFO   | Found global min in 10/14 black-box and all 5 grey-box benchmarks [1912.12637] |

## 6. Theoretical Foundations and Convergence Guarantees

Rigorous convergence proofs for trust-region funnel and optimism-driven Bayesian algorithms have clarified conditions under which iterates achieve feasibility (\(\theta_k \to 0\)), first-order criticality (\(\liminf_{k\to\infty}\|\nabla f(x_k)\|=0\)), and bounded sample regret matching black-box BO up to multiplicative constants determined by problem structure or surrogate accuracy [2511.18998, 2306.05150]. Fully linear/quadratic surrogate models, smoothness/qualification assumptions, and model restoration protocols are essential. Adaptive globalisation mechanisms (single-width funnel vs. multi-entry filters) provide robustness against tuning and initial infeasibility [2511.18998, 2509.01651]. In composite/nested Bayesian schemes, leveraging partial derivative information from white-box mappings further accelerates convergence [2105.04114]. In feedback optimisation controllers, grey-box convex combination yields convergence rates interpolating between model-based and pure model-free bounds as a function of sensitivity accuracy and problem dimension [2404.04355].

## 7. Extensions, Challenges, and Research Directions

Emergent grey-box optimisation research addresses several frontiers:

- Structure generalisation: Multi-sat statistics, instance-difficulty estimators, and local-optima metrics for broader classes of constraint and SAT-like problems [2504.11864];
- High-cardinality variable domains and very large subfunctions: Limiting factor table growth or using neural networks as alternative probabilistic models [1707.03093];
- Multi-objective, constrained, and time-varying extension: Co-occurrence graphs, Pareto-stratum learning, explicit chance-constraint formulations [1707.03093, 2105.04114];
- Hybrid and adaptive surrogate models: Online switching between polynomial, Taylor, and GP models, and robustness against geometry deterioration [2509.01651, 2511.18998, 1912.12637];
- Neural generative models (RBM, DAE, NADE): Nonlinear, high-order dependency capture and integration into EDA sampling strategies [1707.03093].

Open questions remain in automating structural statistic selection, parameter self-tuning (e.g. mask shrinkage rates, surrogate selection criteria), extending discrete combinatorial metrics to real-valued domains, and balancing evolvability/diversity in evolutionary frameworks.

---

Grey-box optimisation occupies a crucial middle ground in computational optimisation, where available analytic structure is maximally leveraged to supplement surrogate-based, data-driven, or evolutionary search, yielding improved sample efficiency, rigorous convergence, and expanded applicability to complex, realistic systems where fully analytic or fully black-box assumptions are untenable.

Source: https://www.emergentmind.com/topics/grey-box-optimisation