Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A sampling criterion for constrained Bayesian optimization with uncertainties (2103.05706v4)

Published 9 Mar 2021 in stat.ML, cs.LG, and math.OC

Abstract: We consider the problem of chance constrained optimization where it is sought to optimize a function and satisfy constraints, both of which are affected by uncertainties. The real world declinations of this problem are particularly challenging because of their inherent computational cost. To tackle such problems, we propose a new Bayesian optimization method. It applies to the situation where the uncertainty comes from some of the inputs, so that it becomes possible to define an acquisition criterion in the joint controlled-uncontrolled input space. The main contribution of this work is an acquisition criterion that accounts for both the average improvement in objective function and the constraint reliability. The criterion is derived following the Stepwise Uncertainty Reduction logic and its maximization provides both optimal controlled and uncontrolled parameters. Analytical expressions are given to efficiently calculate the criterion. Numerical studies on test functions are presented. It is found through experimental comparisons with alternative sampling criteria that the adequation between the sampling criterion and the problem contributes to the efficiency of the overall optimization. As a side result, an expression for the variance of the improvement is given.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Set inversion under functional uncertainties with joint meta-models. preprint available on HAL as document hal-02986558 ( https://hal-ifp.archives-ouvertes.fr/hal-02986558 ), 2020.
  2. Gradient-based simulation optimization under probability constraints. European Journal of Operational Research, 212(2):345–351, 2011.
  3. Studies in linear and non-linear programming, volume 2. Stanford University Press, 1958.
  4. 3rd GECCO workshop for real-parameter optimization. https://coco.gforge.inria.fr/doku.php?id=bbob-2012-results, 2012.
  5. Overview of problem formulations and optimization algorithms in the presence of uncertainty. In Aerospace System Analysis and Optimization in Uncertainty, pages 147–183. Springer, 2020.
  6. Adaptive modeling strategy for constrained global optimization with application to aerodynamic wing design. Aerospace Science and technology, 90:85–102, 2019.
  7. Sequential design of computer experiments for the estimation of a probability of failure. Statistics and Computing, 22(3):773–793, 2012.
  8. Robust optimization, volume 28. Princeton University Press, 2009.
  9. Robust optimization–a comprehensive survey. Computer methods in applied mechanics and engineering, 196(33-34):3190–3218, 2007.
  10. Jean-Marc Bourinet. Reliability analysis and optimal design under uncertainty-Focus on adaptive surrogate-based approaches. Habilitation à diriger les recherches, Univ. Clermont-Auvergne, 2018. https://hal.archives-ouvertes.fr/tel-01737299.
  11. Mirrored sampling and sequential selection for evolution strategies. In PPSN 2010, pages 11–21. Springer, 2010.
  12. Clément Chevalier. Fast uncertainty reduction strategies relying on Gaussian process models. PhD thesis, University of Bern, 2013.
  13. Corrected kriging update formulae for batch-sequential data assimilation. In Mathematics of Planet Earth, pages 119–122. Springer, 2014.
  14. Reliability-based design optimization using kriging surrogates and subset simulation. Structural and Multidisciplinary Optimization, 44(5):673–690, 2011.
  15. AK-MCS: an active learning reliability method combining kriging and monte carlo simulation. Structural Safety, 33(2):145–154, 2011.
  16. Reda El Amri. EFISUR, a new acquisition function. https://github.com/elamrireda/EFISUR, 2021.
  17. Scalable constrained bayesian optimization. International Conference on Artificial Intelligence and Statistics, PMLR, 130:730–738, 2021.
  18. AK-SYS: an adaptation of the AK-MCS method for system reliability. Reliability Engineering & System Safety, 123:137–144, 2014.
  19. Recent advances in robust optimization: An overview. European journal of operational research, 235(3):471–483, 2014.
  20. Roman Garnett. Bayesian Optimization. Cambridge University Press, 2022.
  21. Predictive entropy search for bayesian optimization with unknown constraints. 32nd International Conference on Machine Learning, PMLRnal Conference on Machine Learning, PMLR, 37:1699–1707, 2015.
  22. Uncertainty management for robust industrial design in aeronautics. 2019.
  23. Simultaneous kriging-based estimation and optimization of mean response. Journal of Global Optimization, 2012. DOI 10.1007/s10898-011-9836-5.
  24. Stochastic estimation of the maximum of a regression function. The Annals of Mathematical Statistics, 23(3):462–466, 1952.
  25. Adam: A method for stochastic optimization, 2014.
  26. Rodolphe Le Riche. Optimization under uncertainties: an overview with a focus on Gaussian processes. Lecture at the CNRS French-German University school on Modeling and Numerical Methods for Uncertainty Quantification, cf. https://hal.archives-ouvertes.fr/cel-02285533, September 2019.
  27. Quality through design: Experimental design, off-line quality control, and Taguchi’s contributions. Oxford University Press, USA, 1989.
  28. Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy. In Proc. of the 14th GECCO, pages 321–328. ACM, 2012.
  29. A new expected-improvement algorithm for continuous minimax optimization. Journal of Global Optimization, 64:785–802, 2016.
  30. Quantile-based optimization under uncertainties using bootstrap polynomial chaos expansions. In Proceedings of the 12th International Conference on Structural Safety and Reliability (ICOSSAR 2017), pages 1561–1569. TU Verlag, 2017.
  31. Quantile-based optimization under uncertainties using adaptive kriging surrogate models. Structural and multidisciplinary optimization, 54(6):1403–1421, 2016.
  32. Arkadi Nemirovski. On safe tractable approximations of chance constraints. European Journal of Operational Research, 219(3):707–718, 2012.
  33. Gilles Pagès. Introduction to vector quantization and its applications for numerics. ESAIM: proceedings and surveys, 48:29–79, 2015.
  34. Robust design: An overview. AIAA Journal, 44:181–191, 01 2006.
  35. Efficient global optimization of constrained mixed variable problems. Journal of Global Optimization, 73(3):583–613, 2019.
  36. Constrained bayesian optimization with max-value entropy search. arXiv preprint arXiv:1910.07003, 2019.
  37. Victor Picheny. A Stepwise uncertainty reduction approach to constrained global optimization. In Samuel Kaski and Jukka Corander, editors, Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, volume 33 of Proceedings of Machine Learning Research, pages 787–795, Reykjavik, Iceland, 22–25 Apr 2014. PMLR.
  38. Michael JD Powell. A direct search optimization method that models the objective and constraint functions by linear interpolation. In Advances in optimization and numerical analysis, pages 51–67. Springer, 1994.
  39. Michael JD Powell. The BOBYQA algorithm for bound constrained optimization without derivatives. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge, pages 26–46, 2009.
  40. Optimisation multidisciplinaire en mécanique: Réduction de modèles, robustesse, fiabilité, réalisations logicielles, chapter L’incertitude en conception: formalisation, estimation. Hermes Science Publications, Paris, 2009. (in French).
  41. Spectral representation of robustness measures for optimization under input uncertainty. In ICML2022, the 39th International Conference on Machine Learning, pages 1–26, 2022.
  42. Robust optimization: A kriging-based multi-objective optimization approach. Reliability Engineering and System Safety, 200:106913, 2020.
  43. Group kernels for gaussian process metamodels with categorical inputs. SIAM/ASA Journal on Uncertainty Quantification, 2019.
  44. Global optimization of problems with disconnected feasible regions via surrogate modeling. In 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, page 5573, 2002.
  45. Exploration of metamodeling sampling criteria for constrained global optimization. Engineering optimization, 34(3):263–278, 2002.
  46. Global versus local search in constrained optimization of computer models. Lecture Notes-Monograph Series, pages 11–25, 1998.
  47. Computational methods in optimization considering uncertainties–an overview. Computer Methods in Applied Mechanics and Engineering, 198(1):2–13, 2008.
  48. James C Spall. Introduction to stochastic search and optimization: estimation, simulation, and control, volume 65. John Wiley & Sons, 2005.
  49. X-armed bandits: Optimizing quantiles, CVaR and other risks. In Asian Conference on Machine Learning, pages 252–267. PMLR, 2019.
  50. A review on quantile regression for stochastic computer experiments. Reliability Engineering & System Safety, page 106858, 2020.
  51. A survey on approaches for reliability-based optimization. Structural and Multidisciplinary Optimization, 42(5):645–663, 2010.
  52. Constrained two-step look-ahead bayesian optimization. Advances in Neural Information Processing Systems, 34:12563–12575, 2021.
  53. Analysis of adaptive directional stratification for the controlled estimation of rare event probabilities. Statistics and Computing, 22(3):809–821, 2012.
Citations (11)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now