Training PINNs with Hard Constraints and Adaptive Weights: An Ablation Study

Abstract: Training Physics-Informed Neural Networks (PINNs) to solve stiff time-dependent partial differential equations (PDEs) remains a significant challenge. The main difficulties lie in precisely enforcing initial conditions and balancing the various loss components. In stiff PDEs, where solutions show rapid transitions or sharp gradients, this balance becomes especially difficult. The wide range of scales in the dynamics can cause certain losses to dominate, leading to unstable or inefficient training. This paper presents a comprehensive ablation study focused on two pivotal training schemes: the enforcement of hard constraints for initial and boundary conditions, and the implementation of adaptive weights. We specifically examine their impact on the performance of PINNs when applied to stiff time-dependent PDEs from materials science and mathematical biology applications. We conduct extensive numerical experiments across a diverse range of time-dependent PDEs from Allen-Cahn, Cahn-Hillard, to Gray-Scott systems. We further discuss the implications of our findings for improving the robustness and efficiency of PINN training, particularly in settings where accurate representation of initial conditions and balanced loss contributions are paramount.