The Baldwin Effect in Advancing Generalizability of Physics-Informed Neural Networks (2312.03243v2)

Abstract: Physics-informed neural networks (PINNs) are at the forefront of scientific machine learning, making possible the creation of machine intelligence that is cognizant of physical laws and able to accurately simulate them. However, today's PINNs are often trained for a single physics task and require computationally expensive re-training for each new task, even for tasks from similar physics domains. To address this limitation, this paper proposes a pioneering approach to advance the generalizability of PINNs through the framework of Baldwinian evolution. Drawing inspiration from the neurodevelopment of precocial species that have evolved to learn, predict and react quickly to their environment, we envision PINNs that are pre-wired with connection strengths inducing strong biases towards efficient learning of physics. A novel two-stage stochastic programming formulation coupling evolutionary selection pressure (based on proficiency over a distribution of physics tasks) with lifetime learning (to specialize on a sampled subset of those tasks) is proposed to instantiate the Baldwin effect. The evolved Baldwinian-PINNs demonstrate fast and physics-compliant prediction capabilities across a range of empirically challenging problem instances with more than an order of magnitude improvement in prediction accuracy at a fraction of the computation cost compared to state-of-the-art gradient-based meta-learning methods. For example, when solving the diffusion-reaction equation, a 70x improvement in accuracy was obtained while taking 700x less computational time. This paper thus marks a leap forward in the meta-learning of PINNs as generalizable physics solvers. Sample codes are available at \url{https://github.com/chiuph/Baldwinian-PINN}.

• The paper introduces a Baldwinian meta-learning strategy that pre-wires PINNs for robust physics modeling using evolutionary algorithms.
• The paper demonstrates notable computational speedups and accuracy improvements on linear and nonlinear ODE/PDE problems via a rapid Moore-Penrose pseudoinverse approach.
• The paper highlights the potential of B-PINNs to generalize across diverse physical scenarios, paving the way for efficient and adaptable scientific machine learning models.

Introduction

Physics-informed neural networks (PINNs) epitomize a promising branch of scientific machine learning, aiming to design machine intelligence that integrates physical laws into its learning process. These networks incorporate known physical constraints into the learning model, lending them the capability to make predictions that adhere to these constraints. While PINNs have seen applications across various scientific realms, they face limitations in generalizing to physics scenarios that remain unexplored during their training phase. Retraining or fine-tuning models for new scenarios can be computationally intensive, propelling research into methods that allow PINNs to learn efficiently.

Baldwin Effect and Neural Evolution

The paper proposes a fresh approach to the meta-learning of PINNs via the Baldwin effect, a concept from evolutionary biology suggesting that learned abilities can over time become innate through natural selection. Analogous to the development in certain precocial species that are born with innate skills due to selective pressures, the research explores the possibility of pre-wiring a PINN's architecture for robust learning of physics laws. The paper employs an iterative evolutionary algorithm to optimize the network, leading toward PINNs with connection strengths innately inclined towards modeling physics accurately across varied scenarios.

Computational Advantages

The proposed Baldwinian PINNs (B-PINNs) demonstrated exceptional computational benefits. For several linear and nonlinear ODE/PDE problems representative of real-world phenomena, these networks provided fast and accurate predictions on unseen tasks. The optimization relied on lessening the least-squares learning problem for the output layer, which could be rapidly solved using the Moore-Penrose pseudoinverse. Compared to recent meta-learning PINNs, the B-PINNs presented notable computation speedups and accuracy improvements, signifying substantial advances in modeling dynamical systems and scientific phenomena.

Implications for Machine Learning and Physics

The investigation reveals the potential of B-PINNs to serve as a foundation for scientific machine learning models capable of flexibly and reliably predicting a broad array of physical processes. The evolutionary techniques utilized also illustrate the extensive customization possible for neural networks through non-differentiable selection criteria.

Conclusion

The Baldwinian evolution of PINNs explored in this paper opens up promising avenues for creating generalizable and computationally efficient models in scientific machine learning. This novel approach paves the way for quick and accurate predictions across a spectrum of physics tasks, redefining the capabilities and applications of physics-informed machine learning.