Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations (1811.02033v1)

Abstract: We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve in a unified manner forward, inverse and mixed stochastic problems based on a limited number of scattered measurements. Unlike standard GANs relying only on data for training, here we encoded into the architecture of GANs the governing physical laws in the form of stochastic differential equations (SDEs) using automatic differentiation. In particular, we applied Wasserstein GANs with gradient penalty (WGAN-GP) for its enhanced stability compared to vanilla GANs. We first tested WGAN-GP in approximating Gaussian processes of different correlation lengths based on data realizations collected from simultaneous reads at sparsely placed sensors. We obtained good approximation of the generated stochastic processes to the target ones even for a mismatch between the input noise dimensionality and the effective dimensionality of the target stochastic processes. We also studied the overfitting issue for both the discriminator and generator, and we found that overfitting occurs also in the generator in addition to the discriminator as previously reported. Subsequently, we considered the solution of elliptic SDEs requiring approximations of three stochastic processes, namely the solution, the forcing, and the diffusion coefficient. We used three generators for the PI-GANs, two of them were feed forward deep neural networks (DNNs) while the other one was the neural network induced by the SDE. Depending on the data, we employed one or multiple feed forward DNNs as the discriminators in PI-GANs. Here, we have demonstrated the accuracy and effectiveness of PI-GANs in solving SDEs for up to 30 dimensions, but in principle, PI-GANs could tackle very high dimensional problems given more sensor data with low-polynomial growth in computational cost.

• The paper introduces PI-GANs, a novel framework that embeds physical laws into GANs to tackle complex stochastic differential equations.
• It leverages automatic differentiation and the WGAN-GP technique to model unknown stochastic processes while ensuring training stability.
• The approach demonstrates robust performance on forward, inverse, and mixed SDE problems, even with high-dimensional and multi-source data.

Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

This paper introduces a novel framework called Physics-Informed Generative Adversarial Networks (PI-GANs), devised to solve various stochastic differential equation (SDE) problems. The approach integrates generative adversarial networks (GANs) with domain knowledge expressed through stochastic differential equations, overcoming limitations found in purely data-driven methods. Notably, PI-GANs involve embedding physical laws directly within the learning algorithm. The researchers apply Wasserstein GANs with gradient penalty (WGAN-GP), leveraging their properties for improved stability in training compared to traditional GANs.

Methodology and Implementation

PI-GANs' core innovation lies in encoding physical principles into GANs while preserving the model's flexibility to learn unknown stochastic properties from data. The architecture incorporates automatic differentiation for directly implementing the influence of stochastic differential equations. The paper focuses on solving forward, inverse, and mixed stochastic problems, offering a cohesive solution across these problem types.

The network architecture combines discriminators and a set of generators, where the latter includes deep neural networks (DNNs) to model stochastic processes such as the solution, forcing, and diffusion coefficients. This setup is used for solving elliptic SDEs, a challenging class of problems characterized by diverse input data acquired from sparsely distributed sensors. An essential consideration is the GAN's capacity to handle overfitting efficiently and the paper confirms that overfitting exceptions observed before in discriminators apply similarly to generators, necessitating a thoughtful design of network parameters.

Numerical Analysis and Results

PI-GANs excel in approximating stochastic behavior even when significant discrepancies exist between input noise dimensionality and the effective dimensionality of the target process. Extensive trials involving Gaussian processes show promising results, evidencing PI-GANs' effectiveness in modeling complex stochastic landscapes with lower overfitting risk. Even in significantly high-dimensional cases, PI-GANs demonstrate proficiency, suggesting a potential to scale to very high dimensions efficiently due to their low-polynomial computational cost growth.

One prominent feature of the paper includes extending PI-GANs to scenarios with multiple, unaligned datasets. This capability is rare in current methodologies, enabling the integration of snapshots from different source groups to enhance predictive accuracy. Such adaptability indicates robustness in handling real-world applications where data collection conditions may vary.

Implications and Future Work

The advancement presented in this research provides essential insights and methodologies for addressing stochastic differential equations under data-limited and complexity-constrained conditions. The paper argues for the usability of PI-GANs in engineering applications where noise and data sparsity form significant barriers. Given the flexible and comprehensive framework, future work would involve developing techniques to quantify uncertainty within the generator outputs more explicitly and reduce computational demands further.

Moreover, exploring PI-GANs' scalability to systems characterized by exceptionally high dimensional stochastic processes is a promising direction. These developments could substantiate PI-GANs as a foundational methodology in computational modeling across physics-driven domains, with applications ranging from climate modeling to financial systems.

In conclusion, the paper enriches the toolbox of stochastic modeling by combining adversarial learning with strong physics-informed backgrounds, pushing boundaries on solving high-dimensional SDEs. This integration aligns the GAN's learning capabilities with domain-specific constraints, circumventing the curse of dimensionality often encountered in such problems.