Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A local approach to parameter space reduction for regression and classification tasks (2107.10867v3)

Published 22 Jul 2021 in stat.ML, cs.NA, and math.NA

Abstract: Parameter space reduction has been proved to be a crucial tool to speed-up the execution of many numerical tasks such as optimization, inverse problems, sensitivity analysis, and surrogate models' design, especially when in presence of high-dimensional parametrized systems. In this work we propose a new method called local active subspaces (LAS), which explores the synergies of active subspaces with supervised clustering techniques in order to carry out a more efficient dimension reduction in the parameter space. The clustering is performed without losing the input-output relations by introducing a distance metric induced by the global active subspace. We present two possible clustering algorithms: K-medoids and a hierarchical top-down approach, which is able to impose a variety of subdivision criteria specifically tailored for parameter space reduction tasks. This method is particularly useful for the community working on surrogate modelling. Frequently, the parameter space presents subdomains where the objective function of interest varies less on average along different directions. So, it could be approximated more accurately if restricted to those subdomains and studied separately. We tested the new method over several numerical experiments of increasing complexity, we show how to deal with vectorial outputs, and how to classify the different regions with respect to the local active subspace dimension. Employing this classification technique as a preprocessing step in the parameter space, or output space in case of vectorial outputs, brings remarkable results for the purpose of surrogate modelling.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Theory of wing sections: including a summary of airfoil data. Courier Corporation, 2012.
  2. D. Amsallem and C. Farhat. Interpolation method for adapting reduced-order models and application to aeroelasticity. AIAA journal, 46(7):1803–1813, 2008. doi:10.2514/1.35374.
  3. K. Basu and A. B. Owen. Transformations and Hardy–Krause Variation. SIAM Journal on Numerical Analysis, 54(3):1946–1966, 2016. doi:10.1137/15M1052184.
  4. Model Reduction of Parametrized Systems, volume 17 of MS&A series. Springer, 2017.
  5. Model Reduction Framework with a New Take on Active Subspaces for Optimization Problems with Linearized Fluid-Structure Interaction Constraints. International Journal for Numerical Methods in Engineering, 2020. doi:10.1002/nme.6376.
  6. Active Manifolds: A non-linear analogue to Active Subspaces. In Proceddings of the 36th International Conference on Machine Learning, ICML 2019, pages 764–772, Long Beach, California, USA, 9–15 June 2019.
  7. Data-driven science and engineering: Machine learning, dynamical systems, and control. Cambridge University Press, 2019.
  8. Data-driven aerospace engineering: reframing the industry with machine learning. AIAA Journal, 59(8):2820–2847, 2021. doi:10.2514/1.J060131.
  9. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning, pages 108–122, 2013.
  10. P. Chen and O. Ghattas. Hessian-based sampling for high-dimensional model reduction. International Journal for Uncertainty Quantification, 9(2), 2019. doi:10.1615/Int.J.UncertaintyQuantification.2019028753.
  11. Model reduction methods. In E. Stein, R. de Borst, and T. J. R. Hughes, editors, Encyclopedia of Computational Mechanics, Second Edition, pages 1–36. John Wiley & Sons, Ltd., 2017.
  12. A Short Review on Model Order Reduction Based on Proper Generalized Decomposition. Archives of Computational Methods in Engineering, 18(4):395, 2011. doi:10.1007/s11831-011-9064-7.
  13. P. G. Constantine. Active subspaces: Emerging ideas for dimension reduction in parameter studies, volume 2 of SIAM Spotlights. SIAM, 2015.
  14. Forward and backward uncertainty quantification with active subspaces: application to hypersonic flows around a cylinder. Journal of Computational Physics, 407:109079, 2020. doi:10.1016/j.jcp.2019.109079.
  15. Model order reduction assisted by deep neural networks (ROM-net). Advanced Modeling and Simulation in Engineering Sciences, 7(1):1–27, 2020. doi:10.1186/s40323-020-00153-6.
  16. Hull Shape Design Optimization with Parameter Space and Model Reductions, and Self-Learning Mesh Morphing. Journal of Marine Science and Engineering, 9(2):185, 2021. doi:10.3390/jmse9020185.
  17. A non-intrusive approach for reconstruction of POD modal coefficients through active subspaces. Comptes Rendus Mécanique de l’Académie des Sciences, 347(11):873–881, November 2019. doi:10.1016/j.crme.2019.11.012.
  18. A Supervised Learning Approach Involving Active Subspaces for an Efficient Genetic Algorithm in High-Dimensional Optimization Problems. SIAM Journal on Scientific Computing, 43(3):B831–B853, 2021. doi:10.1137/20M1345219.
  19. A modified SEIR model for the spread of Ebola in Western Africa and metrics for resource allocation. Applied Mathematics and Computation, 324:141–155, 2018. doi:10.1016/j.amc.2017.11.039.
  20. Matrix computations, volume 3. Johns Hopkins University Press, 2013.
  21. GPy. GPy: A Gaussian process framework in Python. http://github.com/SheffieldML/GPy, since 2012.
  22. Data mining: Concepts and techniques. The Morgan Kaufmann Series in Data Management Systems, 5(4):83–124, 2012. doi:10.1016/C2009-0-61819-5.
  23. T. Hastie and R. Tibshirani. Discriminant adaptive nearest neighbor classification. IEEE transactions on pattern analysis and machine intelligence, 18(6):607–616, 1996. doi:10.1109/34.506411.
  24. Wing Design by Numerical Optimization. Journal of Aircraft, 15(7):407–412, 1978. doi:10.2514/3.58379.
  25. The Characteristics of 78 Related Airfoil Sections from Tests in the Variable-Density Wind Tunnel. Technical Report 430, N.A.C.A., 1933.
  26. L. Kaufman and P. J. Rousseeuw. Finding Groups in Data: An Introduction to Cluster Analysis, volume 344 of Wiley Series in Probability and Statistics. John Wiley & Sons, 2005.
  27. K.-C. Li. Sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86(414):316–327, 1991. doi:10.2307/2290563.
  28. A reduced basis element method for the steady Stokes problem. ESAIM: Mathematical Modelling and Numerical Analysis, 40(3):529–552, 2006. doi:10.1051/m2an:2006021.
  29. Active subspaces for shape optimization. In 10th AIAA multidisciplinary design optimization conference, page 1171, 2014.
  30. F. E. Maranzana. On the Location of Supply Points to Minimize Transport Costs. Journal of the Operational Research Society, 15(3):261–270, 1964. doi:10.1057/jors.1964.47.
  31. K. P. Murphy. Machine learning: a probabilistic perspective. MIT press, 2012.
  32. Generalized bounds for active subspaces. Electronic Journal of Statistics, 14(1):917–943, 2020. doi:10.1214/20-EJS1684.
  33. A simple and fast algorithm for K-medoids clustering. Expert Systems with Applications, 36(2):3336–3341, 2009. doi:10.1016/j.eswa.2008.01.039.
  34. A. Pinkus. Ridge functions, volume 205. Cambridge University Press, 2015.
  35. Kernel-based active subspaces with application to computational fluid dynamics parametric problems using discontinuous Galerkin method. International Journal for Numerical Methods in Engineering, 123(23):6000–6027, December 2022. doi:10.1002/nme.7099.
  36. Multi-fidelity data fusion through parameter space reduction with applications to automotive engineering. International Journal for Numerical Methods in Engineering, 124(23):5293–5311, December 2023. doi:10.1002/nme.7349.
  37. ATHENA: Advanced Techniques for High dimensional parameter spaces to Enhance Numerical Analysis. Software Impacts, 10:100133, 2021. doi:10.1016/j.simpa.2021.100133.
  38. Basic Ideas and Tools for Projection-Based Model Reduction of Parametric Partial Differential Equations. In P. Benner, S. Grivet-Talocia, A. Quarteroni, G. Rozza, W. H. A. Schilders, and L. M. Silveira, editors, Model Order Reduction, volume 2, chapter 1, pages 1–47. De Gruyter, Berlin, Boston, 2020. doi:10.1515/9783110671490-001.
  39. Advances in Reduced Order Methods for Parametric Industrial Problems in Computational Fluid Dynamics. In R. Owen, R. de Borst, J. Reese, and P. Chris, editors, Proceedings of the 6th European Conference on Computational Mechanics: Solids, Structures and Coupled Problems, ECCM 2018 and 7th European Conference on Computational Fluid Dynamics, ECFD 2018, pages 59–76, Glasgow, UK, 2020.
  40. E. Schubert and P. J. Rousseeuw. Faster k-medoids clustering: improving the PAM, CLARA, and CLARANS algorithms. In International conference on similarity search and applications, pages 171–187. Springer, 2019. doi:10.1007/978-3-030-32047-8_16.
  41. M. Sugiyama. Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of machine learning research, 8(5), 2007.
  42. T. J. Sullivan. Introduction to Uncertainty Quantification, volume 63. Springer, 2015. doi:10.1007/978-3-319-23395-6.
  43. Combined parameter and model reduction of cardiovascular problems by means of active subspaces and POD-Galerkin methods. In D. Boffi, L. F. Pavarino, G. Rozza, S. Scacchi, and C. Vergara, editors, Mathematical and Numerical Modeling of the Cardiovascular System and Applications, volume 16 of SEMA-SIMAI Series, pages 185–207. Springer International Publishing, 2018. doi:10.1007/978-3-319-96649-6_8.
  44. Enhancing CFD predictions in shape design problems by model and parameter space reduction. Advanced Modeling and Simulation in Engineering Sciences, 7(40), 2020. doi:10.1186/s40323-020-00177-y.
  45. A multi-fidelity approach coupling parameter space reduction and non-intrusive POD with application to structural optimization of passenger ship hulls. International Journal for Numerical Methods in Engineering, 124(5):1193–1210, March 2023. doi:10.1002/nme.7159.
  46. Reduction in Parameter Space. In G. Rozza, G. Stabile, and F. Ballarin, editors, Advanced Reduced Order Methods and Applications in Computational Fluid Dynamics, CS&E Series, chapter 16. SIAM Press, 2022. doi:10.1137/1.9781611977257.ch16.
  47. Dimension reduction in heterogeneous parametric spaces with application to naval engineering shape design problems. Advanced Modeling and Simulation in Engineering Sciences, 5(1):25, Sep 2018. doi:10.1186/s40323-018-0118-3.
  48. J. A. Tropp. User-Friendly Tail Bounds for Sums of Random Matrices. Foundations of computational mathematics, 12(4):389–434, 2012. doi:10.1007/s10208-011-9099-z.
  49. Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning series. MIT press Cambridge, MA, 2006.
  50. Localized sliced inverse regression. Journal of Computational and Graphical Statistics, 19(4):843–860, 2010. doi:10.1198/jcgs.2010.08080.
  51. Clustered active-subspace based local gaussian process emulator for high-dimensional and complex computer models. Journal of Computational Physics, 450:110840, 2022. doi:10.1016/j.jcp.2021.110840.
  52. Gradient-based dimension reduction of multivariate vector-valued functions. SIAM Journal on Scientific Computing, 42(1):A534–A558, 2020. doi:10.1137/18M1221837.
  53. Certified dimension reduction in nonlinear bayesian inverse problems. Mathematics of Computation, 91(336):1789–1835, 2022. doi:10.1090/mcom/3737.
  54. Learning nonlinear level sets for dimensionality reduction in function approximation. In Advances in Neural Information Processing Systems, pages 13199–13208, 2019.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets