Using Distance Correlation for Efficient Bayesian Optimization

Abstract: The need to collect data via expensive measurements of black-box functions is prevalent across science, engineering and medicine. As an example, hyperparameter tuning of a large AI model is critical to its predictive performance but is generally time-consuming and unwieldy. Bayesian optimization (BO) is a collection of methods that aim to address this issue by means of Bayesian statistical inference. In this work, we put forward a BO scheme named BDC, which integrates BO with a statistical measure of association of two random variables called Distance Correlation. BDC balances exploration and exploitation automatically, and requires no manual hyperparameter tuning. We evaluate BDC on a range of benchmark tests and observe that it performs on per with popular BO methods such as the expected improvement and max-value entropy search. We also apply BDC to optimization of sequential integral observations of an unknown terrain and confirm its utility.