Papers
Topics
Authors
Recent
Search
2000 character limit reached

Random Exploration in Bayesian Optimization: Order-Optimal Regret and Computational Efficiency

Published 23 Oct 2023 in cs.LG and stat.ML | (2310.15351v2)

Abstract: We consider Bayesian optimization using Gaussian Process models, also referred to as kernel-based bandit optimization. We study the methodology of exploring the domain using random samples drawn from a distribution. We show that this random exploration approach achieves the optimal error rates. Our analysis is based on novel concentration bounds in an infinite dimensional Hilbert space established in this work, which may be of independent interest. We further develop an algorithm based on random exploration with domain shrinking and establish its order-optimal regret guarantees under both noise-free and noisy settings. In the noise-free setting, our analysis closes the existing gap in regret performance and thereby resolves a COLT open problem. The proposed algorithm also enjoys a computational advantage over prevailing methods due to the random exploration that obviates the expensive optimization of a non-convex acquisition function for choosing the query points at each iteration.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (62)
  1. Structure of level sets and sard-type properties of lipschitz maps. Annali della Scuola Normale Superiore di Pisa. Classe di Scienze. Serie V, 4, 08 2011. doi: 10.2422/2036-2145.201107_006.
  2. Extension of sampling inequalities to Sobolev semi-norms of fractional order and derivative data. Numerische Mathematik, 121(3):587–608, 2012. ISSN 0029599X. doi: 10.1007/s00211-011-0439-3.
  3. Hybrid batch bayesian optimization. In Proceedings of the 29th International Conference on Machine Learning, ICML, volume 2, pages 1215–1222, 2012. ISBN 9781450312851.
  4. Bastian Bohn. Error analysis of regularized and unregularized least-squares regression on discretized function spaces. PhD thesis, Rheinische Friedrich-Wilhelms-Universität Bonn, 2017. URL https://hdl.handle.net/20.500.11811/7094.
  5. B. Bohn. On the convergence rate of sparse grid least squares regression. In Sparse Grids and Applications, pages 19–41. Springer International Publishing, 2018. ISBN 978-3-319-75426-0.
  6. B. Bohn and M. Griebel. Error estimates for multivariate regression on discretized function spaces. SIAM Journal on Numerical Analysis, 55(4):1843–1866, 2017.
  7. The mathematical theory of finite element methods, volume 3. Springer, 2008.
  8. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, 2010.
  9. A. Buchholz. Operator khintchine inequality in non-commutative probability. Mathematische Annalen, 319(1):1–16, 2001.
  10. A. Buchholz. Optimal constants in khintchine type inequalities for fermions, rademachers and q-gaussian operators. Bulletin of The Polish Academy of Sciences Mathematics, 53:315–321, 2005. URL https://api.semanticscholar.org/CorpusID:55683104.
  11. A. D. Bull. Convergence rates of efficient global optimization algorithms. Journal of Machine Learning Research, 12:2879–2904, 2011. ISSN 15324435.
  12. High-Dimensional Experimental Design and Kernel Bandits. In Proceedings of the 38th International Conference on Machine Learning, ICML, 2021. URL https://arxiv.org/abs/2105.05806v1http://arxiv.org/abs/2105.05806.
  13. Online learning with kernel losses. In Proceedings of the 36th International Conference on Machine Learning (ICML), pages 971–980. PMLR, 2019.
  14. Discrete least squares polynomial approximation with random evaluations- application to parametric and stochastic elliptic pdes. ESAIM: Mathematical Modelling and Numerical Analysis-Modélisation Mathématique et Analyse Numérique, 49(3):815–837, 2015.
  15. S. R. Chowdhury and A. Gopalan. On kernelized multi-armed bandits. In Proceedings of the 34th International Conference on Machine Learning, ICML, volume 2, pages 1397–1422, 2017. ISBN 9781510855144.
  16. A. Cohen and G. Migliorati. Optimal weighted least-squares methods. The SIAM journal of computational mathematics, 3:181–203, 2017.
  17. On the stability and accuracy of least squares approximations. Foundations of Computational Mathematics, 13:819–834, 2013.
  18. Exponential regret bounds for Gaussian process bandits with deterministic observations. In Proceedings of the 29th International Conference on Machine Learning, ICML, volume 2, pages 1743–1750, 2012. ISBN 9781450312851.
  19. C. F. Dunkl and Y. Xu. Orthogonal Polynomials of Several Variables. Encyclopedia of Mathematics and its Applications. Cambridge University Press, 2 edition, 2014. doi: 10.1017/CBO9781107786134.
  20. A knowledge-gradient policy for sequential information collection. SIAM Journal on Control and Optimization, 47(5):2410–2439, 2008.
  21. Bayesian optimization for adaptive experimental design: A review. IEEE access, 8:13937–13948, 2020.
  22. K. Gröchenig. Sampling, marcinkiewicz–zygmund inequalities, approximation, and quadrature rules. Journal of Approximation Theory, 257:105455, 2020.
  23. Regret bounds for gaussian process bandit problems. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS), pages 273–280, 2010.
  24. D. R. Jones. A taxonomy of global optimization methods based on response surfaces. Journal of global optimization, 21:345–383, 2001.
  25. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13:455–492, 1998.
  26. Worst-case recovery guarantees for least squares approximation using random samples. Constructive Approximation, 54(2):295–352, 2021.
  27. Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences, 2018.
  28. D. Krieg and M. Ullrich. Function values are enough for l2-approximation. Foundations of Computational Mathematics, 21:1141–1151, 2021a. doi: https://doi.org/10.1007/s10208-020-09481-w.
  29. D. Krieg and M. Ullrich. Function values are enough for l2-approximation: Part ii. Journal of Complexity, 66, 2021b. ISSN 0885-064X. doi: https://doi.org/10.1016/j.jco.2021.101569.
  30. H. Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86:97–106, 1964.
  31. J. Lee. Introduction to Topological Manifolds. Springer, 2010.
  32. Hyperband: A novel bandit-based approach to hyperparameter optimization, 2016.
  33. Z. Li and J. Scarlett. Gaussian process bandit optimization with few batches. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, AISTATS, 2022.
  34. Automatic gait optimization with gaussian process regression. In Proceedings of the 20th International Joint Conference on Artifical Intelligence (IJCAI), volume 7, pages 944–949, 2007.
  35. Efficient batch black-box optimization with deterministic regret bounds, 2020.
  36. J. Močkus. On bayesian methods for seeking the extremum. In Optimization Techniques IFIP Technical Conference, pages 400–404, Berlin, Heidelberg, 1975. Springer Berlin Heidelberg. ISBN 978-3-540-37497-8.
  37. M. Moeller and T. Ullrich. L 2-norm sampling discretization and recovery of functions from rkhs with finite trace. Sampling Theory, Signal Processing, and Data Analysis, 19(2):13, 2021.
  38. Towards Global Optimization, volume 2, chapter The application of Bayesian methods for seeking the extremum, pages 117–129. Elsevier, 09 1978. ISBN 0-444-85171-2.
  39. Sobolev error estimates and a bernstein inequality for scattered data interpolation via radial basis functions. Constructive Approximation, 24:175–186, 2006.
  40. A. M. Ostrowski. A quantitative formulation of slyvester’s law of inertia. Proceedings of the National Academy of Sciences, 45(5):740–744, 1959. doi: 10.1073/pnas.45.5.740. URL https://www.pnas.org/doi/abs/10.1073/pnas.45.5.740.
  41. A benchmark of kriging-based infill criteria for noisy optimization. Structural and Multidisciplinary Optimization, 48(3):607–626, 2013. ISSN 1615147X. doi: 10.1007/s00158-013-0919-4. URL https://link.springer.com/article/10.1007/s00158-013-0919-4.
  42. Practical hilbert space approximate bayesian gaussian processes for probabilistic programming. Statistics and Computing, 33(1):17, 2023.
  43. W. Rudin. Real and complex analysis, 3rd ed. McGraw-Hill, Inc., USA, 1987. ISBN 0070542341.
  44. A domain-shrinking based Bayesian optimization algorithm with order-optimal regret performance. In Proceedings of the 35th Annual Conference on Neural Information Processing Systems, volume 34, 2021.
  45. Collaborative Learning in Kernel-based Bandits for Distributed Users, 2022.
  46. Lower Bounds on Regret for Noisy Gaussian Process Bandit Optimization. In Conference on Learning Theory, volume 65, pages 1–20, 2017.
  47. S. Smale and D.-X. Zhou. Shannon sampling and function reconstruction from point values. Bulletin of The American Mathematical Society, 41:279–306, 2004. doi: 10.1090/S0273-0979-04-01025-0.
  48. Gaussian process optimization in the bandit setting: no regret and experimental design. In Proceedings of the 27th International Conference on Machine Learning, ICML, pages 1015–1022, 2010. ISBN 9781605589077. doi: 10.1109/TIT.2011.2182033.
  49. I. Steinwart and A. Christmann. Support Vector Machines. Springer, 2008. doi: https://doi.org/10.1007/978-0-387-77242-4.
  50. A. Törn and A. Žilinskas. Global Optimization. Springer Berlin, Heidelberg, 1989.
  51. J. A. Tropp. User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12:389–434, 2012.
  52. R. Tuo and W. Wang. Kriging prediction with isotropic matérn correlations: Robustness and experimental designs. The Journal of Machine Learning Research, 21(1):7604–7641, 2020.
  53. S. Vakili. Open problem: Regret bounds for noise-free kernel-based bandits. In Proceedings of 35th Conference on Learning Theory (COLT), volume 178, pages 5624–5629, 2022.
  54. Optimal order simple regret for Gaussian process bandits. In Proceedings of the 35th Annual Conference on Neural Information Processing Systems, 2021a.
  55. On information gain and regret bounds in Gaussian process bandits. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics, AISTATS, 2021b.
  56. Finite-time analysis of kernelised contextual bandits. In Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, UAI, pages 654–663, 2013.
  57. Explore-exploit in top-n recommender systems via gaussian processes. In Proceedings of the 8th ACM Conference on Recommender Systems, pages 225–232, 2014.
  58. E. Vazquez and J. Bect. Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. Journal of Statistical Planning and Inference, 140(11):3088–3095, 2010. ISSN 0378-3758. doi: https://doi.org/10.1016/j.jspi.2010.04.018.
  59. L. Wasserman. Lecture notes on statistical methods for machine learning, 2008. URL https://www.stat.cmu.edu/~larry/=sml/Concentration.pdf.
  60. H. Wendland. Scattered Data Approximation. Cambridge University Press, 2004. doi: 10.1017/CBO9780511617539.
  61. A novel class of stabilized greedy kernel approximation algorithms: Convergence, stability and uniform point distribution. Journal of Approximation Theory, 262, 2021. ISSN 10960430. doi: 10.1016/j.jat.2020.105508.
  62. Convergence guarantees for gaussian process means with misspecified likelihoods and smoothness. The Journal of Machine Learning Research, 22(1):5468–5507, 2021.
Citations (5)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free