Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks (1905.01205v2)

Abstract: One of the open problems in scientific computing is the long-time integration of nonlinear stochastic partial differential equations (SPDEs). We address this problem by taking advantage of recent advances in scientific machine learning and the dynamically orthogonal (DO) and bi-orthogonal (BO) methods for representing stochastic processes. Specifically, we propose two new Physics-Informed Neural Networks (PINNs) for solving time-dependent SPDEs, namely the NN-DO/BO methods, which incorporate the DO/BO constraints into the loss function with an implicit form instead of generating explicit expressions for the temporal derivatives of the DO/BO modes. Hence, the proposed methods overcome some of the drawbacks of the original DO/BO methods: we do not need the assumption that the covariance matrix of the random coefficients is invertible as in the original DO method, and we can remove the assumption of no eigenvalue crossing as in the original BO method. Moreover, the NN-DO/BO methods can be used to solve time-dependent stochastic inverse problems with the same formulation and computational complexity as for forward problems. We demonstrate the capability of the proposed methods via several numerical examples: (1) A linear stochastic advection equation with deterministic initial condition where the original DO/BO method would fail; (2) Long-time integration of the stochastic Burgers' equation with many eigenvalue crossings during the whole time evolution where the original BO method fails. (3) Nonlinear reaction diffusion equation: we consider both the forward and the inverse problem, including noisy initial data, to investigate the flexibility of the NN-DO/BO methods in handling inverse and mixed type problems. Taken together, these simulation results demonstrate that the NN-DO/BO methods can be employed to effectively quantify uncertainty propagation in a wide range of physical problems.

• The paper introduces NN-DO and NN-BO methods that combine PINNs with DO/BO techniques to overcome challenges in solving time-dependent SPDEs.
• The NN-DO method resolves covariance singularity issues while the NN-BO method adeptly manages eigenvalue crossing during stochastic evolution.
• Both approaches demonstrate robust accuracy on benchmark problems, enabling advanced solutions in fields like fluid dynamics and atmospheric science.

Solving Time-Dependent Stochastic PDEs with Physics-Informed Neural Networks

The paper "Learning in Modal Space: Solving Time-Dependent Stochastic PDEs Using Physics-Informed Neural Networks" introduces novel approaches based on Physics-Informed Neural Networks (PINNs) to address the challenges associated with solving time-dependent stochastic partial differential equations (SPDEs). The advancements discussed encompass modifications of the Dynamically Orthogonal (DO) and Bi-Orthogonal (BO) methods, yielding the NN-DO and NN-BO approaches. These methods aim to overcome inherent limitations within the original DO/BO methodologies, namely, the assumptions regarding the covariance matrix invertibility and eigenvalue crossing, thus expanding the scope of solving complex SPDEs with PINNs.

Methodological Contributions

The paper innovatively integrates the capabilities of PINNs with the DO/BO frameworks, presenting two new methodologies:

1. NN-DO Method: This approach implements the dynamically orthogonal conditions within the PINNs framework, offering a solution that sidesteps the singularity issue associated with the covariance matrix in traditional DO methods. Notably, it can be applied effectively to SPDEs with deterministic initial conditions, an area where conventional methods struggle.
2. NN-BO Method: Utilizing the static orthogonality inherent in the BO methods, NN-BO allows for the handling of scenarios with significant eigenvalue crossings during the solution evolution. By incorporating these constraints into the loss function, this approach mitigates the risk of instability typically observed in standard BO formulations under similar conditions.

Both methods employ the generalized Karhunen-Loève (KL) expansion to represent the stochastic processes, thereby formulating a scheme that directly integrates the DO and BO constraints into the machine learning paradigm of the PINNs.

Numerical Results and Applications

The paper provides comprehensive numerical evaluations across several benchmark problems:

• Linear Stochastic Advection Equation: The NN-DO method demonstrates proficiency in scenarios where traditional DO fails, achieving accurate solutions without the need for an invertible covariance matrix from the outset.
• Stochastic Burgers' Equation: Here, NN-BO showcases its strength in managing long-term integrations and complex eigenvalue crossing phenomena, with accuracy maintained over extended temporal domains.
• Nonlinear Reaction-Diffusion Problems: Penetrating deeper, the methods show adaptability in contexts involving noisy initial conditions and can solve stochastic inverse problems, a clear demonstration of the utility in predicting quantifiable uncertainties.

The results underscored are consistent in terms of both accuracy and computational feasibility, positioning these methods as significant additions to tools available for solving high-dimensional stochastic problems.

Implications and Future Directions

This research enriches the theoretical framework of SPDEs and extends practical capabilities by fusing machine learning with rigorous mathematical principles predicated on the DO/BO methodologies. The flexibility to accommodate deterministic and stochastic data, tackle high-dimensional solution spaces, and solve inverse problems signifies substantial implications for physics-informed machine learning applications, particularly in uncertain and complex domains such as fluid dynamics, atmospheric science, and beyond.

However, there remains an on-going challenge in optimizing the efficiency of PINNs, especially concerning computational overheads and the inherent non-convex optimization landscape. Propositions for future exploration encompass leveraging parallel computing strategies, such as employing the parareal algorithm to expedite temporal domain solutions.

In conclusion, the advancements elucidated within this paper foster a deeper integration of machine learning with traditional stochastic analysis, paving the way for substantive developments in robust and scalable computational methods for SPDEs. The NN-DO/BO methods' ability to assimilate real-time data through flexible neural network structures foretells a transformative potential in predictive modeling amidst uncertainty.