Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization (2006.05078v3)

Abstract: In many real-world scenarios, decision makers seek to efficiently optimize multiple competing objectives in a sample-efficient fashion. Multi-objective Bayesian optimization (BO) is a common approach, but many of the best-performing acquisition functions do not have known analytic gradients and suffer from high computational overhead. We leverage recent advances in programming models and hardware acceleration for multi-objective BO using Expected Hypervolume Improvement (EHVI)---an algorithm notorious for its high computational complexity. We derive a novel formulation of q-Expected Hypervolume Improvement (qEHVI), an acquisition function that extends EHVI to the parallel, constrained evaluation setting. qEHVI is an exact computation of the joint EHVI of q new candidate points (up to Monte-Carlo (MC) integration error). Whereas previous EHVI formulations rely on gradient-free acquisition optimization or approximated gradients, we compute exact gradients of the MC estimator via auto-differentiation, thereby enabling efficient and effective optimization using first-order and quasi-second-order methods. Our empirical evaluation demonstrates that qEHVI is computationally tractable in many practical scenarios and outperforms state-of-the-art multi-objective BO algorithms at a fraction of their wall time.

• The paper introduces a novel differentiable q-EHVI method that extends EHVI to parallel settings in multi-objective optimization.
• It computes exact gradients using auto-differentiation, enabling efficient gradient-based optimization.
• Empirical evaluations show enhanced performance and lower computational costs compared to state-of-the-art methods.

Overview of Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization

This paper introduces a novel method for efficiently handling multi-objective Bayesian optimization (BO) in scenarios where several objectives must be optimized simultaneously. Specifically, it focuses on the computationally intensive Expected Hypervolume Improvement (EHVI), a key metric in multi-objective optimization. The authors present a differentiated version of $q$-Expected Hypervolume Improvement ($q$-EHVI) designed for parallel and constrained settings, allowing decision-makers to evaluate multiple samples in parallel, which can significantly expedite the optimization process.

Key Contributions

1. Novel Formulation of $q$-EHVI: The paper derives a new formulation of $q$-EHVI that extends EHVI to parallel settings where multiple designs are tested simultaneously. This extension is crucial for practical applications where evaluation costs are high, and multiple solutions can be assessed concurrently.
2. Exact Gradients via Auto-Differentiation: Previous approaches suffered from either utilizing gradient-free optimization methods or relying on approximate gradients. This work provides a method to compute exact gradients of the Monte Carlo (MC) estimator through auto-differentiation, facilitating efficient gradient-based optimization.
3. Implementation on Modern Hardware: Leveraging modern programming models and hardware acceleration, the authors demonstrate that EHVI becomes computationally tractable and can outperform state-of-the-art methods in various practical scenarios. The results indicate enhanced performance at a fraction of the computational cost.
4. Handling Constraints: The authors extend EHVI to incorporate outcome constraints, increasing its applicability in real-world scenarios where possible solutions must satisfy specific criteria.
5. Empirical Evaluation: Performance comparisons against contemporary methods such as SMS-EGO and PESMO reveal that the proposed method not only offers better optimization performance but also does so with reduced computational overhead.

Implications and Future Work

The introduction of a differentiable $q$-EHVI equipped with exact gradients has practical implications in multi-objective settings, making it possible to handle more complex and larger-scale optimization problems efficiently. The methodology's efficiency strongly aligns with current needs in fields like engineering design, especially in sectors like automotive safety and streaming adaptive control policies.

Theoretically, the paper invites further exploration into the convergence guarantees of the proposed SAA approach within more generalized contexts. In practice, this work prompts the integration of more sophisticated heuristics within hypervolume algorithms for further scaling, potentially enhancing the versatile application of Bayesian optimization techniques.

Conclusion

In summary, the paper makes significant strides toward making multi-objective optimization more accessible and efficient. Through leveraging modern automated differentiation and parallel computing capabilities, it sets a robust foundation for future explorations into scalable, effective multi-objective Bayesian optimization, with potential advancements in both theoretical properties and computational techniques.