Physics-Enhanced Machine Learning: a position paper for dynamical systems investigations (2405.05987v2)

Abstract: This position paper takes a broad look at Physics-Enhanced Machine Learning (PEML) -- also known as Scientific Machine Learning -- with particular focus to those PEML strategies developed to tackle dynamical systems' challenges. The need to go beyond Machine Learning (ML) strategies is driven by: (i) limited volume of informative data, (ii) avoiding accurate-but-wrong predictions; (iii) dealing with uncertainties; (iv) providing Explainable and Interpretable inferences. A general definition of PEML is provided by considering four physics and domain knowledge biases, and three broad groups of PEML approaches are discussed: physics-guided, physics-encoded and physics-informed. The advantages and challenges in developing PEML strategies for guiding high-consequence decision making in engineering applications involving complex dynamical systems, are presented.

• The paper provides an extensive overview of Physics-Enhanced Machine Learning (PEML) for dynamical systems, addressing limitations of data-driven ML like limited data, uncertain predictions, and lack of interpretability.
• It categorizes PEML strategies into three groups based on physics and data integration: Physics-Informed (embedding physics bias), Physics-Guided (updating physics models with data), and Physics-Encoded (explicit hybrid models).
• The paper highlights key challenges and opportunities for PEML in dynamical systems, including developing consistent evaluation metrics, automated error correction, and scalable solutions for complex problems.

The paper "Physics-Enhanced Machine Learning: a position paper for dynamical systems investigations" by Alice Cicirello provides an extensive overview of Physics-Enhanced Machine Learning (PEML), also known as Scientific Machine Learning (SciML), with a targeted application to challenges in dynamical systems. PEML approaches strive to overcome the limitations of traditional Machine Learning (ML) by integrating physics and domain knowledge, addressing issues such as limited data, the risk of accurate-but-wrong predictions, uncertainties, and the need for explainable and interpretable inferences.

Key Points and Concepts

Challenges of Data-Driven ML:

1. Limited Data Availability: In many engineering domains, data can be sparse, noisy, and expensive to acquire, limiting the reliability of purely data-driven models.
2. Accurate-but-Wrong Predictions: Relying solely on data correlations without accounting for causality may result in models that are unable to generalize to unseen conditions and are prone to physically inconsistent predictions.
3. Dealing with Uncertainties: Both data-driven and pure physics-based models are challenged by various uncertainties, necessitating hybrid approaches to improve prediction accuracy and robustness.
4. Explainability and Interpretability: PEML seeks to combine the insights gained from data-driven models with the transparency of physics-based models for enhanced decision-making.

PEML Framework:

The paper categorizes PEML strategies into three groups based on how they integrate physics, data, and biases:

1. Physics-Informed ML: Relies on medium to large volumes of data, embedding physics biases within the ML process to restrict the solution space to feasible regions. This category includes techniques like Physics-Informed Neural Networks (PINNs) and hybrid approaches in Gaussian Processes.
2. Physics-Guided ML: Utilizes detailed physics-based models and small to medium datasets. The emphasis is on updating model parameters for accurate predictions, typical of probabilistic model updating strategies where biases are primarily guided by domain-specific knowledge.
3. Physics-Encoded ML: Involves explicit combinations of data-driven and physics-based models, incorporating biases to learn missing physical dynamics and uncertain parameters. Examples include Latent Force Models and hybrid system identification approaches.

Types of Biases in PEML:

• Observational Bias: Involves using data reflecting underlying physical laws, which can include synthetic data to supplement real measurements.
• Learning Bias: Encompasses refinement of the learning process, including algorithm selection, loss functions, and constraints.
• Inductive Bias: Integrates prior assumptions, such as conservation laws and symmetries, into the model architecture.
• Model Form/Discrepancy Bias: Leverages partial knowledge of physical models to inform the learning process, essential for dynamical systems with known governing equations.

Open Challenges and Opportunities

The paper discusses several challenges facing PEML developments, particularly in the context of dynamical systems:

• Development of consistent metrics for evaluating PEML prediction quality, benchmarks for performance comparison, and validation strategies.
• Automated identification and correction of errors in data, model biases, or model architecture.
• Scalable solutions capable of addressing complex, multi-scale problems, enhancing the interpretability and practical application of PEML models.

The paper underscores that while there is no one-size-fits-all PEML strategy, the choice of approach should depend heavily on the specific problem physics, available data, and the purposes of the user. The synthesis of computational science with ML provides unique opportunities to guide high-consequence decision-making in engineering, advancing the capabilities of dynamical system analyses.