Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization (2405.15285v1)
Abstract: Local Bayesian optimization is a promising practical approach to solve the high dimensional black-box function optimization problem. Among them is the approximated gradient class of methods, which implements a strategy similar to gradient descent. These methods have achieved good experimental results and theoretical guarantees. However, given the distributional properties of the Gaussian processes applied on these methods, there may be potential to further exploit the information of the Gaussian processes to facilitate the BO search. In this work, we develop the relationship between the steps of the gradient descent method and one that minimizes the Upper Confidence Bound (UCB), and show that the latter can be a better strategy than direct gradient descent when a Gaussian process is applied as a surrogate. Through this insight, we propose a new local Bayesian optimization algorithm, MinUCB, which replaces the gradient descent step with minimizing UCB in GIBO. We further show that MinUCB maintains a similar convergence rate with GIBO. We then improve the acquisition function of MinUCB further through a look ahead strategy, and obtain a more efficient algorithm LA-MinUCB. We apply our algorithms on different synthetic and real-world functions, and the results show the effectiveness of our method. Our algorithms also illustrate improvements on local search strategies from an upper bound perspective in Bayesian optimization, and provides a new direction for future algorithm design.
- Local bayesian optimization of motor skills. In International Conference on Machine Learning, pages 41–50. PMLR, 2017.
- Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in neural information processing systems, 33:21524–21538, 2020.
- Random search for hyper-parameter optimization. Journal of machine learning research, 13(2), 2012.
- A survey on high-dimensional gaussian process modeling with application to bayesian optimization. ACM Transactions on Evolutionary Learning and Optimization, 2(2):1–26, 2022.
- High-dimensional gaussian process bandits. Advances in neural information processing systems, 26, 2013.
- High-dimensional bayesian optimization with sparse axis-aligned subspaces. In Uncertainty in Artificial Intelligence, pages 493–503. PMLR, 2021.
- Scalable global optimization via local bayesian optimization. Advances in neural information processing systems, 32, 2019.
- Peter I Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811, 2018.
- Discovering and exploiting additive structure for bayesian optimization. In Artificial Intelligence and Statistics, pages 1311–1319. PMLR, 2017.
- Bayesian optimization with inequality constraints. In ICML, volume 2014, pages 937–945, 2014.
- Bayesian optimization with unknown constraints. arXiv preprint arXiv:1403.5607, 2014.
- Bayesian optimization for chemical reactions. Chimia, 77(1/2):31–38, 2023.
- High dimensional bayesian optimisation and bandits via additive models. In International conference on machine learning, pages 295–304. PMLR, 2015.
- Advances in bayesian optimization with applications in aerospace engineering. In 2018 AIAA Non-Deterministic Approaches Conference, page 1656, 2018.
- Uniform error bounds for gaussian process regression with application to safe control. Advances in Neural Information Processing Systems, 32, 2019.
- Re-examining linear embeddings for high-dimensional bayesian optimization. Advances in neural information processing systems, 33:1546–1558, 2020.
- High dimensional bayesian optimization via restricted projection pursuit models. In Artificial Intelligence and Statistics, pages 884–892. PMLR, 2016.
- Simple random search of static linear policies is competitive for reinforcement learning. Advances in neural information processing systems, 31, 2018.
- Local latent space bayesian optimization over structured inputs. Advances in neural information processing systems, 35:34505–34518, 2022.
- High-dimensional bayesian optimization using low-dimensional feature spaces. Machine Learning, 109:1925–1943, 2020.
- Local policy search with bayesian optimization. Advances in Neural Information Processing Systems, 34:20708–20720, 2021.
- A framework for bayesian optimization in embedded subspaces. In International Conference on Machine Learning, pages 4752–4761. PMLR, 2019.
- Local bayesian optimization via maximizing probability of descent. Advances in neural information processing systems, 35:13190–13202, 2022.
- Significance of gradient information in bayesian optimization. In International Conference on Artificial Intelligence and Statistics, pages 2836–2844. PMLR, 2021.
- Proxybo: Accelerating neural architecture search via bayesian optimization with zero-cost proxies. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9792–9801, 2023.
- Wei Shyy. A study of finite difference approximations to steady-state, convection-dominated flow problems. Journal of Computational Physics, 57(3):415–438, 1985.
- Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25, 2012.
- Gaussian process optimization in the bandit setting: No regret and experimental design. arXiv preprint arXiv:0912.3995, 2009.
- Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012.
- Robust model-free reinforcement learning with multi-objective bayesian optimization. In 2020 IEEE international conference on robotics and automation (ICRA), pages 10702–10708. IEEE, 2020.
- Think global and act local: Bayesian optimisation over high-dimensional categorical and mixed search spaces. arXiv preprint arXiv:2102.07188, 2021.
- Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55:361–387, 2016.
- Bayesian optimization with gradients. Advances in neural information processing systems, 30, 2017.
- The behavior and convergence of local bayesian optimization. Advances in Neural Information Processing Systems, 36, 2024.