Additive Multi-Index Gaussian process modeling, with application to multi-physics surrogate modeling of the quark-gluon plasma (2306.07299v1)
Abstract: The Quark-Gluon Plasma (QGP) is a unique phase of nuclear matter, theorized to have filled the Universe shortly after the Big Bang. A critical challenge in studying the QGP is that, to reconcile experimental observables with theoretical parameters, one requires many simulation runs of a complex physics model over a high-dimensional parameter space. Each run is computationally very expensive, requiring thousands of CPU hours, thus limiting physicists to only several hundred runs. Given limited training data for high-dimensional prediction, existing surrogate models often yield poor predictions with high predictive uncertainties, leading to imprecise scientific findings. To address this, we propose a new Additive Multi-Index Gaussian process (AdMIn-GP) model, which leverages a flexible additive structure on low-dimensional embeddings of the parameter space. This is guided by prior scientific knowledge that the QGP is dominated by multiple distinct physical phenomena (i.e., multiphysics), each involving a small number of latent parameters. The AdMIn-GP models for such embedded structures within a flexible Bayesian nonparametric framework, which facilitates efficient model fitting via a carefully constructed variational inference approach with inducing points. We show the effectiveness of the AdMIn-GP via a suite of numerical experiments and our QGP application, where we demonstrate considerably improved surrogate modeling performance over existing models.
- Alver, B. (2007). System size, energy, pseudorapidity, and centrality dependence of elliptic flow. Phys. Rev. Lett., 98:242302.
- Collision geometry fluctuations and triangular flow in heavy-ion collisions. Phys. Rev. C, 81:054905. [Erratum: Phys.Rev.C 82, 039903 (2010)].
- Gaussian predictive process models for large spatial data sets. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 70(4):825–848.
- Sequential dimension reduction for learning features of expensive black-box functions. Preprint at https://hal.science/hal-01688329v2/file/main.pdf.
- Bayesian estimation of the specific shear and bulk viscosity of quark–gluon plasma. Nature Physics, 15(11):1113–1117.
- Dirichlet–Laplace priors for optimal shrinkage. Journal of the American Statistical Association, 110(512):1479–1490.
- Replication or exploration? Sequential design for stochastic simulation experiments. Technometrics, 61(1):7–23.
- Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518):859–877.
- Buckingham, E. (1914). On physically similar systems; illustrations of the use of dimensional equations. Physical Review, 4(4):345.
- Determining the jet transport coefficient q^^𝑞\hat{q}over^ start_ARG italic_q end_ARG from inclusive hadron suppression measurements using Bayesian parameter estimation. Physical Review C, 104(2):024905.
- Bayes and Empirical Bayes Methods for Data Analysis. Springer.
- Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences. The Annals of Statistics, 40(4):2069 – 2101.
- Projection pursuit Gaussian process regression. IISE Transactions. To appear.
- Function-on-function kriging, with applications to three-dimensional printing of aortic tissues. Technometrics, 63:384–395.
- A hierarchical expected improvement method for Bayesian optimization. Journal of the American Statistical Association, to appear.
- Chikuse, Y. (2003). Statistics on Special Manifolds, volume 1. Springer.
- Active subspace methods in theory and practice: applications to kriging surfaces. SIAM Journal on Scientific Computing, 36(4):A1500–A1524.
- Discovering an active subspace in a single-diode solar cell model. Statistical Analysis and Data Mining: The ASA Data Science Journal, 8(5):264–273.
- Bayesian prediction of deterministic functions, with applications to the design and analysis of computer experiments. Journal of the American Statistical Association, 86(416):953–963.
- Deep Gaussian processes. In Carvalho, C. M. and Ravikumar, P., editors, Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, pages 207–215.
- Randomly projected additive Gaussian processes for regression. Proceedings of the 37th International Conference on Machine Learning, 119:2453–2463.
- Additive covariance kernels for high-dimensional Gaussian process modeling. Annales de la Faculté des sciences de Toulouse : Mathématiques, 21(3):481–499.
- Additive Gaussian processes. In Advances in Neural Information Processing Systems, volume 24.
- Bayesian analysis of QGP jet transport using multi-scale modeling applied to inclusive hadron and reconstructed jet data. arXiv preprint arXiv:2208.07950.
- High-dimensional Bayesian optimization with sparse axis-aligned subspaces. In Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, pages 493–503. PMLR. ISSN: 2640-3498.
- Everett, D. (2021). Quantifying the Quark Gluon Plasma. The Ohio State University.
- Multisystem Bayesian constraints on the transport coefficients of QCD matter. Physical Review C, 103(5):054904.
- Phenomenological constraints on the transport properties of QCD matter with data-driven model averaging. Physical Review Letters, 126(24):242301.
- Role of bulk viscosity in deuteron production in ultrarelativistic nuclear collisions. Physical Review C, 106(6):064901.
- Hydrodynamic modeling of heavy-ion collisions. Int. J. Mod. Phys. A, 28:1340011.
- Mapping the hydrodynamic response to the initial geometry in heavy-ion collisions. Phys. Rev. C, 85:024908.
- GPytorch: Blackbox matrix-matrix Gaussian process inference with GPU acceleration. In Advances in Neural Information Processing Systems, volume 31.
- Scaling multidimensional Gaussian processes using projected additive approximations. In International Conference on Machine Learning, pages 454–461. PMLR.
- Dimension reduction in magnetohydrodynamics power generation models: Dimensional analysis and active subspaces. Statistical Analysis and Data Mining: The ASA Data Science Journal, 10(5):312–325.
- Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378.
- GPy (since 2012). GPy: A Gaussian process framework in Python. http://github.com/SheffieldML/GPy.
- Gramacy, R. (2020). Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences. Chapman Hall/CRC.
- Gramacy, R. B. (2007). tgp: an R package for Bayesian nonstationary, semiparametric nonlinear regression and design by treed Gaussian process models. Journal of Statistical Software, 19:1–46.
- Gaussian process single-index models as emulators for computer experiments. Technometrics, 54(1):30–41.
- Prediction of standard enthalpies of formation based on hydrocarbon molecular descriptors and active subspace methodology. Industrial & Engineering Chemistry Research, 59(10):4785–4791.
- On almost linearity of low dimensional projections from high dimensional data. The Annals of Statistics, 21(2):867 – 889.
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction, volume 2. Springer.
- Collective flow and viscosity in relativistic heavy-ion collisions. Ann. Rev. Nucl. Part. Sci., 63:123–151.
- Active learning of multi-index function models. In Pereira, F., Burges, C., Bottou, L., and Weinberger, K., editors, Advances in Neural Information Processing Systems, volume 25.
- Hersbach, H. (2000). Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather and Forecasting, 15:559–570.
- Stochastic structured variational inference. In Lebanon, G. and Vishwanathan, S. V. N., editors, International Conference on Artificial Intelligence and Statistics, pages 361–369. PMLR.
- Stochastic variational inference. Journal of Machine Learning Research, 14(40):1303–1347.
- Bayesian quantile regression for single-index models. Statistics and Computing, 23:437–454.
- A graphical Gaussian process model for multi-fidelity emulation of expensive computer codes. arXiv preprint arXiv:2108.00306.
- Conglomerate multi-fidelity Gaussian process modeling, with application to heavy-ion collisions. arXiv preprint arXiv:2209.13748.
- Variable selection consistency of Gaussian process regression. The Annals of Statistics, 49(5):2491–2505.
- Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492.
- High dimensional Bayesian optimisation and bandits via additive models. In Bach, F. and Blei, D., editors, International Conference on Machine Learning, pages 295–304. PMLR.
- Adam: A method for stochastic optimization. In International Conference on Learning Representations.
- Inclusive jet and hadron suppression in a multistage approach. Physical Review C, 107(3):034911.
- Re-examining linear embeddings for high-dimensional Bayesian optimization. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., editors, Advances in Neural Information Processing Systems, pages 1546–1558.
- Variable selection for Gaussian process models in computer experiments. Technometrics, 48(4):478–490.
- Efficient emulation of relativistic heavy ion collisions with transfer learning. Physical Review C, 105(3):034910.
- Initial state fluctuations and final state correlations in relativistic heavy-ion collisions. J. Phys. G, 41:063102.
- An efficient surrogate model for emulation and physics extraction of large eddy simulations. J Amer Statist Assoc, 113(524):1443–1456.
- A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 42(1):55–61.
- Eccentricity fluctuations and its possible effect on elliptic flow measurements. arXiv preprint nucl-ex/0312008.
- Alternative ansatz to wounded nucleon and binary collision scaling in high-energy nuclear collisions. Phys. Rev. C, 92(1):011901.
- Determining fundamental properties of matter created in ultrarelativistic heavy-ion collisions. Physical Review C, 89(3):034917.
- Ollitrault, J.-Y. (1992). Anisotropy as a signature of transverse collective flow. Phys. Rev. D, 46:229–245.
- Variable selection for Gaussian processes via sensitivity analysis of the posterior predictive distribution. In International Conference on Artificial Intelligence and Statistics, pages 1743–1752.
- Automatic Differentiation in PyTorch. NIPS 2017 Workshop Autodiff.
- Constraining the initial state granularity with bulk observables in Au+Au collisions at GeV. Journal of Physics G: Nuclear and Particle Physics, 38(4):045102.
- Pinkus, A. (2015). Ridge Functions, volume 205. Cambridge University Press.
- Occam’s razor. In Advances in Neural Information Processing Systems. MIT Press.
- Variational Bayes for high-dimensional linear regression with sparse priors. Journal of the American Statistical Association, 117(539):1270–1281.
- Design and analysis of computer experiments. Statistical Science, 4(4):409–423.
- Toward a deeper understanding of how experiments constrain the underlying physics of heavy-ion collisions. Physical Review C, 93(2):024908.
- The Design and Analysis of Computer Experiments. Springer.
- Vecchia-approximated deep Gaussian processes for computer experiments. Journal of Computational and Graphical Statistics. To appear.
- Polynomial ridge flowfield estimation. Physics of Fluids, 33(12):127110.
- Dimension reduction via Gaussian ridge functions. SIAM/ASA Journal on Uncertainty Quantification, 7(4):1301–1322.
- Sparse Gaussian processes using pseudo-inputs. In Advances in Neural Information Processing Systems. MIT Press.
- Variable noise and dimensionality reduction for sparse Gaussian processes. arXiv preprint arXiv:1206.6873.
- Stein, M. L. (1999). Interpolation of Spatial Data: Some Theory for Kriging. Springer Science & Business Media.
- A generalized gaussian process model for computer experiments with binary time series. Journal of the American Statistical Association, 115(530):945–956.
- Bayesian Gaussian process latent variable model. In International Conference on Artificial Intelligence and Statistics, pages 844–851.
- Gaussian processes with built-in dimensionality reduction: Applications to high-dimensional uncertainty propagation. Journal of Computational Physics, 321:191–223.
- Empirical Processes in M-Estimation, volume 6. Cambridge University Press.
- Rates of contraction of posterior distributions based on Gaussian process priors. The Annals of Statistics, 36(3):1435–1463.
- Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research, 55(1):361–387.
- Screening, predicting, and computer experiments. Technometrics, 34(1):15–25.
- Xia, Y. (2008). A multiple-index model and dimension reduction. Journal of the American Statistical Association, 103(484):1631–1640.
- Bayesian uncertainty quantification for low-rank matrix completion. Bayesian Analysis, 18(2):491–518.