Multi-view Bayesian optimisation in reduced dimension for engineering design (2501.01552v1)

Abstract: Bayesian optimisation is an adaptive sampling strategy for constructing a Gaussian process surrogate to emulate a black-box computational model with the aim of efficiently searching for the global minimum. However, Gaussian processes have limited applicability for engineering problems with many design variables. Their scalability can be significantly improved by identifying a low-dimensional vector of latent variables that serve as inputs to the Gaussian process. In this paper, we introduce a multi-view learning strategy that considers both the input design variables and output data representing the objective or constraint functions, to identify a low-dimensional space of latent variables. Adopting a fully probabilistic viewpoint, we use probabilistic partial least squares (PPLS) to learn an orthogonal mapping from the design variables to the latent variables using training data consisting of inputs and outputs of the black-box computational model. The latent variables and posterior probability densities of the probabilistic partial least squares and Gaussian process models are determined sequentially and iteratively, with retraining occurring at each adaptive sampling iteration. We compare the proposed probabilistic partial least squares Bayesian optimisation (PPLS-BO) strategy to its deterministic counterpart, partial least squares Bayesian optimisation (PLS-BO), and classical Bayesian optimisation, demonstrating significant improvements in convergence to the global minimum.

• The paper introduces a novel PPLS-BO methodology that integrates multi-view learning to reduce design space dimensionality.
• It leverages Gaussian process surrogates and adaptive sampling to significantly improve convergence rates in complex engineering scenarios.
• Comparative results demonstrate that PPLS-BO outperforms traditional optimisation methods, enhancing both theoretical and practical design outcomes.

Multi-view Bayesian Optimisation in Reduced Dimension for Engineering Design

The paper "Multi-view Bayesian Optimisation in Reduced Dimension for Engineering Design" by Archbold, Kazlauskaite, and Cirak addresses the challenges associated with Bayesian optimisation in high-dimensional spaces, particularly in the context of engineering design. Bayesian optimisation, known for its sample efficiency, often faces scalability issues in problems with a multitude of design variables due to the "curse of dimensionality." This paper proposes a novel framework that integrates multi-view learning strategies into the Bayesian optimisation process to mitigate these issues.

Contribution and Methodology

The authors focus on reducing the dimensionality of the design space by identifying a set of latent variables through a multi-view learning approach, specifically probabilistic partial least squares (PPLS). This approach effectively creates a low-dimensional approximation while retaining the critical information necessary for optimisation. By considering both the input design variables and output data, the framework captures more nuanced relationships that may not be evident in traditional methods.

The PPLS model provides a fully probabilistic understanding of the design space, leveraging a Gaussian process (GP) as the surrogate model. The framework is designed to iteratively update as new data becomes available through adaptive sampling, refining the mapping between design variables and latent variables. The combination of PPLS with Bayesian optimisation forms the core of the proposed methodology, denoted as PPLS-BO.

Results and Comparisons

The paper presents a detailed comparison of the proposed PPLS-BO methodology against Partial Least Squares Bayesian Optimisation (PLS-BO) and classical Bayesian optimisation. It demonstrates significant improvements in convergence rates toward the global minimum, particularly in problems characterised by high-dimensional input spaces. The examples provided range from illustrative cases to complex engineering scenarios, such as the optimisation of a cantilever beam design and a welded assembly, demonstrating the versatility and effectiveness of the approach.

In terms of numerical results, the PPLS-BO consistently outperforms its deterministic counterparts, particularly in scenarios where the intrinsic dimensionality of the problem is well captured by the latent space. In one of the examples, the proposed method converges towards the global optimum considerably faster than classical BO and PLS-BO, reaffirming the value of capturing epistemic uncertainties and refining the design space exploratively.

Theoretical Implications and Future Directions

Theoretically, this work enhances the applicability of Bayesian optimisation to complex, real-world engineering design problems by allowing the optimisation to effectively navigate reduced-dimensional spaces. The integration of multi-view learning provides a robust mechanism to capture relevant information across different QOIs, making it a valuable addition to the field of design optimisation.

Looking forward, the PPLS-BO framework highlights the potential for further research into probabilistic modelling techniques that can better integrate into the Bayesian optimisation loop. There is an opportunity to explore other types of surrogates, perhaps those that deviate from Gaussian processes when dealing with non-smooth or discrete landscapes. Additionally, advancements might focus on adaptive strategies for determining the optimal dimensionality of the latent space, enhancing the robustness and applicability of the approach across even more diverse problem sets.

In conclusion, the paper provides a sophisticated and promising approach to handling high-dimensional optimisation challenges in engineering design through an innovative amalgamation of multi-view learning and Bayesian optimisation techniques. The methodological advancements presented carry substantial implications for computational tractability and precision in finding optimal solutions within extensive design spaces.