Papers
Topics
Authors
Recent
Search
2000 character limit reached

SMT 2.0: A Surrogate Modeling Toolbox with a focus on Hierarchical and Mixed Variables Gaussian Processes

Published 23 May 2023 in cs.LG, math.OC, stat.CO, and cs.MS | (2305.13998v5)

Abstract: The Surrogate Modeling Toolbox (SMT) is an open-source Python package that offers a collection of surrogate modeling methods, sampling techniques, and a set of sample problems. This paper presents SMT 2.0, a major new release of SMT that introduces significant upgrades and new features to the toolbox. This release adds the capability to handle mixed-variable surrogate models and hierarchical variables. These types of variables are becoming increasingly important in several surrogate modeling applications. SMT 2.0 also improves SMT by extending sampling methods, adding new surrogate models, and computing variance and kernel derivatives for Kriging. This release also includes new functions to handle noisy and use multifidelity data. To the best of our knowledge, SMT 2.0 is the first open-source surrogate library to propose surrogate models for hierarchical and mixed inputs. This open-source software is distributed under the New BSD license.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (86)
  1. Dakota, a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis: Version 6.13 user’s manual. Technical report, Sandia National Lab.(SNL-NM), Albuquerque, NM (United States), 2020.
  2. A general mathematical framework for constrained mixed-variable blackbox optimization problems with meta and categorical variables. Operations Research Forum, 4:1–37, 2023.
  3. Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in neural information processing systems, 33:21524–21538, 2020.
  4. Disciplinary proper orthogonal decomposition and interpolation for the resolution of parameterized multidisciplinary analysis. International Journal for Numerical Methods in Engineering, 123:3594–3626, 2022.
  5. Numerical methods for the discretization of random fields by means of the karhunen–loève expansion. Computer Methods in Applied Mechanics and Engineering, 271:109–129, 2014.
  6. M. A. Bouhlel and J. Martins. Gradient-enhanced kriging for high-dimensional problems. Engineering with Computers, 35:157–173, 2019.
  7. An improved approach for estimating the hyperparameters of the kriging model for high-dimensional problems through the partial least squares method. Mathematical Problems in Engineering, 2016:6723410, 2016.
  8. Efficient global optimization for high-dimensional constrained problems by using the kriging models combined with the partial least squares method. Engineering Optimization, 50:2038–2053, 2018.
  9. A python surrogate modeling framework with derivatives. Advances in Engineering Software, 135:102662, 2019.
  10. Scalable gradient-enhanced artificial neural networks for airfoil shape design in the subsonic and transonic regimes. Structural and Multidisciplinary Optimization, 61:1363–1376, 2020.
  11. System architecture design space exploration: An approach to modeling and optimization. In AIAA AVIATION 2020 FORUM, 2020.
  12. Effectiveness of surrogate-based optimization algorithms for system architecture optimization. In AIAA AVIATION 2021 FORUM, 2021.
  13. Disciplinary surrogates for gradient-based optimization of multidisciplinary systems. In ECCOMAS Aerobest, 2023.
  14. Trying to elicit and assign goals to the right actors. In Conceptual Modeling: 41st International Conference, ER 2022, 2022.
  15. ParMOO: A Python library for parallel multiobjective simulation optimization. Journal of Open Source Software, 8:4468, 2023.
  16. Multi-fidelity bayesian optimization strategy applied to overall drone design. In AIAA SciTech 2023 Forum, 2023a.
  17. Towards a multi-fidelity and multi-objective bayesian optimization efficient algorithm. Aerospace Science and Technology, 142:108673, 2023b.
  18. Surrogate modeling for capacity planning of charging station equipped with photovoltaic panel and hydropneumatic energy storage. Journal of Energy Resources Technology, 142:050907, 2020.
  19. Trust region based mode pursuing sampling method for global optimization of high dimensional design problems. Journal of Mechanical Design, 137:021407, 2015.
  20. Basic enhancement strategies when using bayesian optimization for hyperparameter tuning of deep neural networks. IEEE access, 8:52588–52608, 2020.
  21. Multi-fidelity gaussian process model for CFD and wind tunnel data fusion. In ECCOMAS Aerobest, 2021.
  22. HEBO: Heteroscedastic Evolutionary Bayesian Optimisation, 2020.
  23. A comparison of mixed-variables bayesian optimization approaches. Advanced Modeling and Simulation in Engineering Sciences, 9:1–29, 2021.
  24. Additive gaussian process for computer models with qualitative and quantitative factors. Technometrics, 59:283–292, 2017.
  25. Multi-fidelity algorithm for the sensitivity analysis of multidisciplinary problems. Journal of Mechanical Design, 145:1–22, 2023.
  26. Periodic version of the minimax distance criterion for monte carlo integration. Advances in Engineering Software, 149:102900, 2020.
  27. Review on python toolboxes for kriging surrogate modelling. In ESREL, 2022.
  28. Automated hybrid propulsion model construction for conceptual aircraft design and optimization. In 33rd Congress of the International Council of the Aeronautical Sciences, ICAS 2022, 2022.
  29. Active learning of linear embeddings for gaussian processes. In Uncertainty in Artificial Intelligence - Proceedings of the 30th Conference, 2013.
  30. E. C. Garrido-Merchán and D. Hernández-Lobato. Dealing with categorical and integer-valued variables in bayesian optimization with gaussian processes. Neurocomputing, 380:20–35, 2020.
  31. Kriging Is Well-Suited to Parallelize Optimization, pages 131–162. Springer Berlin Heidelberg, 2010.
  32. A surrogate modeling and adaptive sampling toolbox for computer based design. Journal of Machine Learning Research, 11:2051–2055, 2010.
  33. M. Halstrup. Black-Box Optimization of Mixed Discrete-Continuous Optimization Problems. PhD thesis, TU Dortmund, 2016.
  34. Bayesian optimization using deep gaussian processes with applications to aerospace system design. Optimization and Engineering, 22:321–361, 2021.
  35. Surrogates for hierarchical search spaces: The wedge-kernel and an automated analysis. In Proceedings of the Genetic and Evolutionary Computation Conference, 2019.
  36. Design and analysis of computer experiments with branching and nested factors. Technometrics, 51:354–365, 2009.
  37. F. Hutter and M. A. Osborne. A kernel for hierarchical parameter spaces, 2013.
  38. J. T. Hwang and J. R. R. A. Martins. A fast-prediction surrogate model for large datasets. Aerospace Science and Technology, 75:74–87, 2018.
  39. Effectively using multifidelity optimization for wind turbine design. Wind Energy Science, 7:991–1006, 2022.
  40. Openbox: A python toolkit for generalized black-box optimization, 2023.
  41. An efficient algorithm for constructing optimal design of computer experiments. Journal of Statistical Planning and Inference, 2:545–554, 2005.
  42. D. R. Jones. A taxonomy of global optimization methods based on response surfaces. Journal of Global Optimization, 21:345–383, 2001.
  43. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13:455–492, 1998.
  44. Tuning hyperparameters without grad students: Scalable and robust bayesian optimisation with dragonfly. Journal of Machine Learning Research, 21:3098–3124, 2020.
  45. Fem based robust design optimization with agros and ārtap. Computers & Mathematics with Applications, 81:618–633, 2021.
  46. Continuous surrogate-based optimization algorithms are well-suited for expensive discrete problems. In Artificial Intelligence and Machine Learning, 2021.
  47. M. Kennedy and A. O’Hagan. Bayesian calibration of computer models. J.R. Statist. Soc. B, 63:425–464, 2001.
  48. Coaxial-injector surrogate modeling based on reynolds-averaged navier–stokes simulations using deep learning. Journal of Propulsion and Power, 38:783–798, 2022.
  49. J. Kudela and R. Matousek. Recent advances and applications of surrogate models for finite element method computations: a review. Soft Computing, 26:13709–13733, 2022.
  50. Numba: A llvm-based python jit compiler. In Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, 2015.
  51. R. Lafage. egobox, a Rust toolbox for efficient global optimization. Journal of Open Source Software, 7:4737, 2022.
  52. Uqlab 2.0 and uqcloud: open-source vs. cloud-based uncertainty quantification. In SIAM Conference on Uncertainty Quantification (SIAM UQ 2022), 2022.
  53. H. Lee. Gaussian Processes, pages 575–577. Springer Berlin Heidelberg, 2011.
  54. SMAC3: A versatile Bayesian optimization package for hyperparameter optimization. Journal of Machine Learning Research, 23:1–9, 2022.
  55. Multioutput gaussian processes with functional data: A study on coastal flood hazard assessment. Reliability Engineering & System Safety, 218:108139, 2022.
  56. ADjoint: An approach for the rapid development of discrete adjoint solvers. AIAA Journal, 46:863–873, 2008.
  57. Engineering design optimization. Cambridge University Press, 2021.
  58. Multi-fidelity efficient global optimization: Methodology and application to airfoil shape design. In AIAA AVIATION 2019 FORUM, 2019.
  59. Variance based sensitivity analysis for monte carlo and importance sampling reliability assessment with gaussian processes. Structural Safety, 93:102116, 2021.
  60. Deep gaussian process emulation using stochastic imputation. Technometrics, 0:1–12, 2022.
  61. SO-MI: A surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Computers & Operations Research, 40:1383–1400, 2013.
  62. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
  63. Efficient global optimization of constrained mixed variable problems. Journal of Global Optimization, 73:583–613, 2019.
  64. Bayesian optimization of variable-size design space problems. Optimization and Engineering, 22:387–447, 2021.
  65. Trieste: Efficiently exploring the depths of black-box functions with tensorflow, 2023.
  66. A systematic exploration of reservoir computing for forecasting complex spatiotemporal dynamics. Neural Networks, 153:530–552, 2022.
  67. High-dimensional efficient global optimization using both random and supervised embeddings. In AIAA AVIATION 2023 Forum, 2023.
  68. Parameterizing correlations: a geometric interpretation. IMA Journal of Management Mathematics, 18:55–73, 2007.
  69. R. Rebonato and P. Jaeckel. The most general methodology to create a valid correlation matrix for risk management and option pricing purposes. Journal of Risk, 2:17–27, 2001.
  70. Dicekriging, diceoptim: Two r packages for the analysis of computer experiments by kriging-based metamodeling and optimization. Journal of statistical software, 51:1–55, 2012.
  71. Group kernels for gaussian process metamodels with categorical inputs. SIAM Journal on Uncertainty Quantification, 8:775–806, 2020.
  72. An efficient parallel global optimization strategy based on kriging properties suitable for material parameters identification. Archive of Mechanical Engineering, 67, 2020.
  73. A mixed integer efficient global optimization algorithm with multiple infill strategy - applied to a wing topology optimization problem. In AIAA SciTech 2019 Forum, 2019.
  74. A mixed-categorical data-driven approach for prediction and optimization of hybrid discontinuous composites performance. In AIAA AVIATION 2022 Forum, 2022.
  75. An adaptive data-driven modelling and optimization framework for complex chemical process design. Computer Aided Chemical Engineering, 48:73–78, 2020.
  76. Bayesian optimization for mixed variables using an adaptive dimension reduction process: applications to aircraft design. In AIAA SciTech 2022 Forum, 2022.
  77. A mixed-categorical correlation kernel for gaussian process. Neurocomputing, 550:126472, 2023.
  78. Derivative-free mixed binary necklace optimization for cyclic-symmetry optimal design problems. Optimization and Engineering, 2021.
  79. Transpiration cooling of high pressure turbine vane with optimized porosity distribution. Applied Thermal Engineering, 223:119831, 2023.
  80. N. Wildberger. A rational approach to trigonometry. Math Horizons, 15:16–20, 2007.
  81. Gaussian processes for machine learning. MIT press Cambridge, MA, 2006.
  82. H. Wold. Soft modelling by latent variables: The non-linear iterative partial least squares (nipals) approach. Journal of Applied Probability, 12:117–142, 1975.
  83. M. Zaefferer and D. Horn. A first analysis of kernels for kriging-based optimization in hierarchical search spaces, 2018.
  84. A latent variable approach to gaussian process modeling with qualitative and quantitative factors. Technometrics, 62:291–302, 2020.
  85. A simple approach to emulation for computer models with qualitative and quantitative factors. Technometrics, 53:266–273, 2011.
  86. M. M. Zuniga and D. Sinoquet. Global optimization for mixed categorical-continuous variables based on gaussian process models with a randomized categorical space exploration step. INFOR: Information Systems and Operational Research, 58:310–341, 2020.
Citations (42)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up