Grey-Box Optimisation: Methods & Applications

• Grey-box optimisation is a computational method that combines analytical (glass-box) and simulation-based (black-box) components to exploit partial structural knowledge.
• It employs trust-region strategies, surrogate modelling, and Bayesian techniques to achieve faster convergence and reduce the number of expensive evaluations.
• Practical implementations demonstrate significant efficiency gains and robust performance in engineering, process systems, and combinatorial optimisation applications.

Grey-box optimisation encompasses computational frameworks wherein the objective function and/or constraints consist of both analytically tractable (glass-box) components and black-box models lacking explicit derivatives. These scenarios arise pervasively in engineering, process systems, simulation-based design, and combinatorial optimisation, reflecting realistic solver settings where partial structural, algebraic, and gradient information is available but key parts remain opaque. By systematically exploiting known model structure—either via analytic derivatives, problem decompositions, or mechanistic knowledge—grey-box optimisation algorithms attain marked improvements in sample efficiency, convergence guarantees, and scalability relative to black-box approaches, while retaining generality and robustness against model uncertainty.

1. Problem Formulation and Modelling Structure

The canonical grey-box problem is:

$$\min_{x\in\mathcal X}\;f(x) \;=\;\min_{x\in\mathcal X}\;[f_g(x) + f_b(x)] \quad \text{s.t.}\quad c(x)=0$$

where:

• $f_g(x)$ represents the glass-box (analytic, differentiable) component,
• $f_b(x)$ captures the black-box (non-analytic, simulation or data-driven) part, approximated via local surrogates,
• $c(x)$ can mix glass-box and black-box algebraic or simulation-based constraints.

Explicit decoupling via variable partitioning and auxiliary outputs is standard, e.g., splitting $x = [w, y, z]$ with $y = t(w)$ representing simulator responses or black-box outputs, and $z$ denoting purely glass-box variables (Hameed et al., 24 Nov 2025). In combinatorial and pseudo-Boolean settings, the objective often decomposes as a sum over localized subfunctions:

$$f(\mathbf{x}) = \sum_{i=1}^{m} f_{i}(\mathbf{x}_{C_i})$$

with each $f_{i}$ acting only on a small mask $C_i$ of variables, and the variable interaction graph (VIG) encoding dependency structure (Piatek et al., 16 Apr 2025, Santana, 2017). In Bayesian optimisation, grey-box models arise when the system performance $f(x)$ is expressible as compositions of known and unknown mappings, e.g., $f(x, y(x))$ where $y$ is GP-modelled (González et al., 1 Jan 2025), or nested multi-level chains (Xu et al., 2023).

2. Algorithmic Approaches: Trust-Region and Surrogate Modelling

Deterministic grey-box optimisation in continuous domains typically exploits trust-region-based strategies (Hameed et al., 24 Nov 2025, Hameed et al., 1 Sep 2025, Sampaio, 2019). At each iterate, glass-box components are treated with exact or linearised differentials, while black-box portions are locally approximated by surrogates. Standard surrogate forms include:

• Polynomial interpolation (linear, quadratic)
• Taylor expansions (if gradients available)
• Gaussian-process regression (GP)
• Hybrid combinations (e.g., Taylor + GP-residual)

A typical trust-region subproblem is:

$$\min_{d}\; f_g(x_k + d) + m_b^k(d) \quad \text{s.t.}\;\|d\|\le\Delta_k,\; h(x_k+d)=0,\; g(x_k+d)\le0$$

with $m_b^k(d)$ constructed to satisfy fully-linear or fully-quadratic error bounds in the trust-region (Hameed et al., 24 Nov 2025).

Globalisation and step acceptance are performed via filter methods—infeasibility/objective pairs—or funnel mechanisms tracking a single scalar error bound (surrogate mismatch or constraint infeasibility), as in the trust-region funnel algorithm (Hameed et al., 24 Nov 2025). New variants incorporate second-order information (local Hessian projections) in trust-region geometries, yielding dramatic reductions in black-box calls and enhanced robustness against tuning parameter choices (Hameed et al., 1 Sep 2025). Restoration phases and surrogate management (geometry repair/poisedness maintenance) further refine trajectory selection and ensure convergence (Sampaio, 2019).

3. Bayesian Grey-Box Optimisation: Composite and Nested Settings

Bayesian frameworks extend grey-box optimisation to situations where expensive-to-evaluate constituents are modelled with Gaussian processes, and compositional or multi-fidelity structure is leveraged (Astudillo et al., 2022, Xu et al., 2023, González et al., 1 Jan 2025, Paulson et al., 2021). Key domains include:

• Composite objectives: $f(x)=g(h(x))$, with known link $g$ and black-box vector-valued $h(x)$;
• Multi-fidelity settings: querying cheaper approximations $h(x,z)$ at varying fidelity parameter $z$;
• Nested and multi-level models: cascading chains $f(x)=e^T\Phi_{m-1}(\Phi_{m-2}(\dots\Phi_0(x)))$ encompassing white- and black-box stages.

GP surrogates are maintained separately for each unknown function, propagating uncertainty through known mappings to compute posterior moments, expected improvement, or lower/upper confidence bounds (González et al., 1 Jan 2025). Notably, the BOIS algorithm achieves analytical forms for acquisition functions by adaptive linearisation around GP means, avoiding expensive sampling or high-dimensional auxiliary optimisation (González et al., 1 Jan 2025). Optimism-driven UCB strategies (solving a constrained auxiliary problem over plausible GP confidence intervals for all black-box components) attain regret bounds matching black-box BO up to a constant, but frequently improve empirical sample efficiency by orders of magnitude (Xu et al., 2023, Paulson et al., 2021). Derivative information in glass-box parts is exploited analytically for faster acquisition maximisation and tighter uncertainty quantification (Paulson et al., 2021).

4. Discrete, Combinatorial, and Evolutionary Grey-Box Methods

In discrete optimisation, grey-box techniques enhance local search and evolutionary algorithms by exploiting known subfunction structure and variable interdependencies (Piatek et al., 16 Apr 2025, Santana, 2017). Mechanisms include:

• Partitioning variable interaction graphs to form local masks for complex, nontrivial moves (including tunnelling across local optima at high Hamming distance);
• Mask-informed perturbation and crossover (e.g. Partition Crossover in scale-free MaxSat instances);
• Auxiliary metrics such as clause-satisfiability (multi-sat statistics) to guide long “wormhole” hops between disconnected regions near critical phase transitions (Piatek et al., 16 Apr 2025);
• Factorised Distribution Algorithms (FDA): Probabilistic graphical models with variable factorisation directly inherited from the problem structure, enabling exact or message-passing sampling on junction trees (Santana, 2017);
• Hybrid grey-box–EDA approaches, moderating building-block disruption and recombination via interplay with graphical/structural models.

These schemes attain polynomial complexity on low tree-width instances and mitigate bottlenecks in searching hard combinatorial landscapes.

5. Practical Implementations and Performance Benchmarks

Open-source frameworks and solvers have codified grey-box optimisation algorithms for scalable application:

• Pyomo-based trust-region funnel method: modular surrogates, criticality/compatibility checks via small LP/QP, analytic derivatives via autodiff, restoration and parameter experimentation (Hameed et al., 24 Nov 2025);
• DEFT-FUNNEL (Matlab): globalisation by clustering-based multistart (MLSL), self-correcting surrogate geometry, slack variable embedding, and practical input handling for white-box derivatives (Sampaio, 2019);
• TRF-Solver: Hessian-enhanced trust-region filter algorithm with surrogate switching, implemented in Pyomo with IPOPT (Hameed et al., 1 Sep 2025);
• Bayesian grey-box optimisation algorithms (BOIS, COBALT): analytical and SAA-based acquisition optimisation, automated derivative exploitation, and explicit chance-constraint handling (González et al., 1 Jan 2025, Paulson et al., 2021).

Benchmark studies indicate marked efficiency gains in iteration counts, black-box evaluations, and CPU time compared to classical or black-box solvers; for instance, order-of-magnitude reductions in black-box calls, and robust convergence to KKT points in large-scale process flowsheet optimisation (Hameed et al., 24 Nov 2025, Hameed et al., 1 Sep 2025, Sampaio, 2019). In combinatorial settings, multi-satisfiability-guided search recovers global connectivity and 100% success on hard MaxSat instances beyond the capability of prior state-of-the-art optimisers (Piatek et al., 16 Apr 2025).

Algorithm/Method	Domain	Efficiency Result
TR funnel (Pyomo)	Process NLPs	2-10x fewer black-box calls vs. filter; strong globalisation (Hameed et al., 24 Nov 2025)
Hessian-enhanced TRF	Process NLPs	5-10x fewer evals; 100% success rate in benchmarks (Hameed et al., 1 Sep 2025)
BOIS	Bayesian BO	Equivalent/better than OP-BO, with 3x less acquisition time (González et al., 1 Jan 2025)
COBALT	Bayesian BO	1-3 orders faster regret reduction; 50% fewer constraint violations (Paulson et al., 2021)
MOCSM (multi-sat)	Max3Sat	100% success in hard instances (<2 min); scales to 350 variables (Piatek et al., 16 Apr 2025)
DEFT-FUNNEL	Global DFO	Found global min in 10/14 black-box and all 5 grey-box benchmarks (Sampaio, 2019)

6. Theoretical Foundations and Convergence Guarantees

Rigorous convergence proofs for trust-region funnel and optimism-driven Bayesian algorithms have clarified conditions under which iterates achieve feasibility ($\theta_k \to 0$), first-order criticality ($\liminf_{k\to\infty}\|\nabla f(x_k)\|=0$), and bounded sample regret matching black-box BO up to multiplicative constants determined by problem structure or surrogate accuracy (Hameed et al., 24 Nov 2025, Xu et al., 2023). Fully linear/quadratic surrogate models, smoothness/qualification assumptions, and model restoration protocols are essential. Adaptive globalisation mechanisms (single-width funnel vs. multi-entry filters) provide robustness against tuning and initial infeasibility (Hameed et al., 24 Nov 2025, Hameed et al., 1 Sep 2025). In composite/nested Bayesian schemes, leveraging partial derivative information from white-box mappings further accelerates convergence (Paulson et al., 2021). In feedback optimisation controllers, grey-box convex combination yields convergence rates interpolating between model-based and pure model-free bounds as a function of sensitivity accuracy and problem dimension (He et al., 5 Apr 2024).

7. Extensions, Challenges, and Research Directions

Emergent grey-box optimisation research addresses several frontiers:

• Structure generalisation: Multi-sat statistics, instance-difficulty estimators, and local-optima metrics for broader classes of constraint and SAT-like problems (Piatek et al., 16 Apr 2025);
• High-cardinality variable domains and very large subfunctions: Limiting factor table growth or using neural networks as alternative probabilistic models (Santana, 2017);
• Multi-objective, constrained, and time-varying extension: Co-occurrence graphs, Pareto-stratum learning, explicit chance-constraint formulations (Santana, 2017, Paulson et al., 2021);
• Hybrid and adaptive surrogate models: Online switching between polynomial, Taylor, and GP models, and robustness against geometry deterioration (Hameed et al., 1 Sep 2025, Hameed et al., 24 Nov 2025, Sampaio, 2019);
• Neural generative models (RBM, DAE, NADE): Nonlinear, high-order dependency capture and integration into EDA sampling strategies (Santana, 2017).

Open questions remain in automating structural statistic selection, parameter self-tuning (e.g. mask shrinkage rates, surrogate selection criteria), extending discrete combinatorial metrics to real-valued domains, and balancing evolvability/diversity in evolutionary frameworks.

---

Grey-box optimisation occupies a crucial middle ground in computational optimisation, where available analytic structure is maximally leveraged to supplement surrogate-based, data-driven, or evolutionary search, yielding improved sample efficiency, rigorous convergence, and expanded applicability to complex, realistic systems where fully analytic or fully black-box assumptions are untenable.