Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Towards Practical Preferential Bayesian Optimization with Skew Gaussian Processes (2302.01513v2)

Published 3 Feb 2023 in cs.LG

Abstract: We study preferential Bayesian optimization (BO) where reliable feedback is limited to pairwise comparison called duels. An important challenge in preferential BO, which uses the preferential Gaussian process (GP) model to represent flexible preference structure, is that the posterior distribution is a computationally intractable skew GP. The most widely used approach for preferential BO is Gaussian approximation, which ignores the skewness of the true posterior. Alternatively, Markov chain Monte Carlo (MCMC) based preferential BO is also proposed. In this work, we first verify the accuracy of Gaussian approximation, from which we reveal the critical problem that the predictive probability of duels can be inaccurate. This observation motivates us to improve the MCMC-based estimation for skew GP, for which we show the practical efficiency of Gibbs sampling and derive the low variance MC estimator. However, the computational time of MCMC can still be a bottleneck in practice. Towards building a more practical preferential BO, we develop a new method that achieves both high computational efficiency and low sample complexity, and then demonstrate its effectiveness through extensive numerical experiments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Efficient simulation for tail probabilities of Gaussian random fields. In 2008 Winter Simulation Conference, pages 328–336. IEEE, 2008.
  2. A. Azzalini. The Skew-Normal and Related Families. Institute of Mathematical Statistics Monographs. Cambridge University Press, 2013.
  3. Skew Gaussian processes for classification. Machine Learning, 109(9):1877–1902, 2020.
  4. Preferential Bayesian optimisation with skew Gaussian processes. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 1842–1850, 2021a.
  5. A unified framework for closed-form nonparametric regression, classification, preference and mixed problems with skew Gaussian processes. Machine Learning, 110(11):3095–3133, 2021b.
  6. J. Breslaw. Random sampling from a truncated multivariate normal distribution. Applied Mathematics Letters, 7(1):1–6, 1994. ISSN 0893-9659.
  7. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv.1012.2599, 2010.
  8. W. Chu and Z. Ghahramani. Extensions of Gaussian processes for ranking: semisupervised and active learning. In Proceedings of the Learning to Rank workshop at the 18th Conference on Neural Information Processing Systems, pages 29–34, 2005a.
  9. W. Chu and Z. Ghahramani. Preference learning with Gaussian processes. In Proceedings of the 22nd international conference on Machine learning, pages 137–144, 2005b.
  10. T. Fauvel and M. Chalk. Efficient exploration in binary and preferential Bayesian optimization. arXiv preprint arXiv:2110.09361, 2021.
  11. Hierarchical decompositions for the computation of high-dimensional multivariate normal probabilities. Journal of Computational and Graphical Statistics, pages 268–277, 2017.
  12. A. Genz. Numerical computation of multivariate normal probabilities. Journal of Computational and Graphical Statistics, 1:141–150, 1992.
  13. Integrals over Gaussians under linear domain constraints. In International Conference on Artificial Intelligence and Statistics, pages 2764–2774. PMLR, 2020.
  14. J. Geweke. Efficient simulation from the multivariate normal and student-t distributions subject to linear constraints and the evaluation of constraint probabilities. In Computing science and statistics: Proceedings of the 23rd symposium on the interface, pages 571–578, 1991.
  15. Preferential Bayesian optimization. In International Conference on Machine Learning, pages 1282–1291. PMLR, 2017.
  16. Predictive entropy search for efficient global optimization of black-box functions. In Advances in Neural Information Processing Systems 27, page 918–926. Curran Associates, Inc., 2014.
  17. W. C. Horrace. Some results on the multivariate truncated normal distribution. Journal of Multivariate Analysis, 94(1):209–221, 2005.
  18. D. Kahneman and A. Tversky. Prospect theory: An analysis of decision under risk. In Handbook of the fundamentals of financial decision making: Part I, pages 99–127. World Scientific, 2013.
  19. Gibbs sampling approach for generation of truncated multivariate Gaussian random variables. In 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No. 99CH36258), volume 3, pages 1757–1760. IEEE, 1999.
  20. Sequential gallery for interactive visual design optimization. ACM Transactions on Graphics (TOG), 39(4):88–1, 2020.
  21. M. Kuss and C. E. Rasmussen. Assessing approximate inference for binary Gaussian process classification. Journal of Machine Learning Research, 6(57):1679–1704, 2005.
  22. Y. Li and S. K. Ghosh. Efficient sampling methods for truncated multivariate normal and student-t distributions subject to linear inequality constraints. Journal of Statistical Theory and Practice, 9(4):712–732, 2015.
  23. D. J. MacKay. Bayesian methods for backpropagation networks. In Models of neural networks III, pages 211–254. Springer, 1996.
  24. T. P. Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, pages 362–369. Morgan Kaufmann Publishers Inc., 2001.
  25. The application of Bayesian methods for seeking the extremum. Towards Global Optimization, 2(117-129):2, 1978.
  26. H. Nickisch and C. E. Rasmussen. Approximations for binary Gaussian process classification. Journal of Machine Learning Research, 9(67):2035–2078, 2008.
  27. Perception-based personalization of hearing aids using Gaussian processes and active learning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23(1):162–173, 2015.
  28. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, 2005.
  29. C. P. Robert. Simulation of truncated normal variables. Statistics and computing, 5(2):121–125, 1995.
  30. D. Russo and B. Van Roy. Learning to optimize via posterior sampling. Mathematics of Operations Research, 39(4):1221–1243, 2014.
  31. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1):148–175, 2016.
  32. Preferential batch Bayesian optimization. In 2021 IEEE 31st International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE, 2021.
  33. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems 25, pages 2951–2959. Curran Associates, Inc., 2012.
  34. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on International Conference on Machine Learning, pages 1015–1022. Omnipress, 2010.
  35. Multi-dueling bandits with dependent arms. In Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence. AUAI Press, 2017.
  36. Advancements in dueling bandits. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 5502–5510. International Joint Conferences on Artificial Intelligence Organization, 2018.
  37. Randomized Gaussian process upper confidence bound with tighter Bayesian regret bounds. In Proceedings of the 40th International Conference on Machine Learning, Proceedings of Machine Learning Research. PMLR, 2023. To appear.
  38. Z. Wang and S. Jegelka. Max-value entropy search for efficient Bayesian optimization. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pages 3627–3635. PMLR, 2017.
  39. The K-armed dueling bandits problem. Journal of Computer and System Sciences, 78(5):1538–1556, 2012.
  40. Interactive exploration-exploitation balancing for generative melody composition. In 26th International Conference on Intelligent User Interfaces, pages 43–47, 2021.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Shion Takeno (18 papers)
  2. Masahiro Nomura (56 papers)
  3. Masayuki Karasuyama (30 papers)
Citations (11)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now