PI-CoF: A Bilevel Optimization Framework for Solving Active Learning Problems using Physics-Information (2402.13588v1)
Abstract: Physics informed neural networks (PINNs) have recently been proposed as surrogate models for solving process optimization problems. However, in an active learning setting collecting enough data for reliably training PINNs poses a challenge. This study proposes a broadly applicable method for incorporating physics information into existing ML models of any type. The proposed method - referred to as PI-CoF for Physics-Informed Correction Factors - introduces additive or multiplicative correction factors for pointwise inference, which are identified by solving a regularized unconstrained optimization problem for reconciliation of physics information and ML model predictions. When ML models are used in an optimization context, using the proposed approach translates into a bilevel optimization problem, where the reconciliation problem is solved as an inner problem each time before evaluating the objective and constraint functions of the outer problem. The utility of the proposed approach is demonstrated through a numerical example, emphasizing constraint satisfaction in a safe Bayesian optimization (BO) setting. Furthermore, a simulation study is carried out by using PI-CoF for the real-time optimization of a fuel cell system. The results show reduced fuel consumption and better reference tracking performance when using the proposed PI-CoF approach in comparison to a constrained BO algorithm not using physics information.
- G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, and L. Yang, “Physics-informed machine learning,” Nature Reviews Physics, vol. 3, no. 6, pp. 422–440, 2021.
- S. Cuomo, V. S. Di Cola, F. Giampaolo, G. Rozza, M. Raissi, and F. Piccialli, “Scientific machine learning through physics–informed neural networks: Where we are and what’s next,” Journal of Scientific Computing, vol. 92, no. 3, p. 88, 2022.
- R. Misener and L. Biegler, “Formulating data-driven surrogate models for process optimization,” Computers & Chemical Engineering, vol. 179, p. 108411, 2023.
- E. S. Koksal and E. Aydin, “Physics informed piecewise linear neural networks for process optimization,” Computers & Chemical Engineering, vol. 174, p. 108244, 2023.
- T. Würth, C. Krauß, C. Zimmerling, and L. Kärger, “Physics-informed neural networks for data-free surrogate modelling and engineering optimization–an example from composite manufacturing,” Materials & Design, vol. 231, p. 112034, 2023.
- E. A. del Rio Chanona, P. Petsagkourakis, E. Bradford, J. A. Graciano, and B. Chachuat, “Real-time optimization meets Bayesian optimization and derivative-free optimization: A tale of modifier adaptation,” Computers & Chemical Engineering, vol. 147, p. 107249, 2021.
- A. Kudva, F. Sorourifar, and J. A. Paulson, “Constrained robust Bayesian optimization of expensive noisy black-box functions with guaranteed regret bounds,” AIChE Journal, vol. 68, no. 12, p. e17857, 2022.
- B. S. Korkmaz, M. Zagórowska, and M. Mercangöz, “Safe optimization of an industrial refrigeration process using an adaptive and explorative framework,” IFAC-PapersOnLine, vol. 56, no. 2, pp. 1400–1405, 2023.
- D. Krishnamoorthy and F. J. Doyle, “Model-free real-time optimization of process systems using safe Bayesian optimization,” AIChE Journal, vol. 69, no. 4, p. e17993, 2023.
- W. J. Tubbs and M. Mercangöz, “Using prior knowledge to improve adaptive real time exploration and optimization,” in 2023 IEEE 21st International Conference on Industrial Informatics (INDIN). IEEE, 2023, pp. 1–8.
- F. Häse, M. Aldeghi, R. J. Hickman, L. M. Roch, and A. Aspuru-Guzik, “Gryffin: An algorithm for Bayesian optimization of categorical variables informed by expert knowledge,” Applied Physics Reviews, vol. 8, no. 3, 2021.
- A. Hanuka, X. Huang, J. Shtalenkova, D. Kennedy, A. Edelen, Z. Zhang, V. Lalchand, D. Ratner, and J. Duris, “Physics model-informed Gaussian process for online optimization of particle accelerators,” Physical Review Accelerators and Beams, vol. 24, no. 7, p. 072802, 2021.
- M. A. Ziatdinov, A. Ghosh, and S. V. Kalinin, “Physics makes the difference: Bayesian optimization and active learning via augmented Gaussian process,” Machine Learning: Science and Technology, vol. 3, no. 1, p. 015003, 2022.
- D. Long, Z. Wang, A. Krishnapriyan, R. Kirby, S. Zhe, and M. Mahoney, “AutoIP: A united framework to integrate physics into Gaussian processes,” in International Conference on Machine Learning. PMLR, 2022, pp. 14 210–14 222.
- X. Yang, D. Barajas-Solano, G. Tartakovsky, and A. M. Tartakovsky, “Physics-informed CoKriging: A Gaussian-process-regression-based multifidelity method for data-model convergence,” Journal of Computational Physics, vol. 395, pp. 410–431, 2019.
- R. Astudillo and P. Frazier, “Bayesian optimization of composite functions,” in International Conference on Machine Learning. PMLR, 2019, pp. 354–363.
- J. A. Paulson and C. Lu, “COBALT: COnstrained Bayesian optimizAtion of computationally expensive grey-box models exploiting derivaTive information,” Computers & Chemical Engineering, vol. 160, p. 107700, 2022.
- L. D. González and V. M. Zavala, “BOIS: Bayesian optimization of interconnected systems,” arXiv preprint arXiv:2311.11254, 2023.
- B. S. Korkmaz, M. Zagórowska, and M. Mercangöz, “Safe and adaptive decision-making for optimization of safety-critical systems: The ARTEO algorithm,” arXiv preprint arXiv:2211.05495, 2022.
- C. König, M. Turchetta, J. Lygeros, A. Rupenyan, and A. Krause, “Safe and efficient model-free adaptive control via Bayesian optimization,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE Press, 2021, p. 9782–9788.
- J. Kupecki, “Off-design analysis of a micro-CHP unit with solid oxide fuel cells fed by DME,” International Journal of Hydrogen Energy, vol. 40, no. 35, pp. 12 009–12 022, 2015.
- B. Chachuat, B. Srinivasan, and D. Bonvin, “Adaptation strategies for real-time optimization,” Computers & Chemical Engineering, vol. 33, no. 10, pp. 1557–1567, 2009.