Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles (1810.13075v2)

Abstract: This paper proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improve the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physical models, while generating outputs consistent with physical laws, and achieving good generalizability. Standard RNNs, even when producing superior prediction accuracy, often produce physically inconsistent results and lack generalizability. We further enhance this approach by using a pre-training method that leverages the simulated data from a physics-based model to address the scarcity of observed data. The PGRNN has the flexibility to incorporate additional physical constraints and we incorporate a density-depth relationship. Both enhancements further improve PGRNN performance. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where mechanistic (also known as process-based) models are used, e.g., power engineering, climate science, materials science, computational chemistry, and biomedicine.

• The paper introduces a novel physics-guided RNN that embeds physical constraints into recurrent neural networks for simulating lake temperature dynamics.
• It demonstrates improved accuracy and stability compared to traditional RNNs by harmonizing empirical data with thermal dynamics principles.
• The approach offers scalable applications in environmental modeling and forecasting, underscoring its potential for broader dynamical system simulations.

Analysis of Sparse Elimination Algorithm and the Use of m-Trees

This paper provides a detailed examination of the nonnumerical complexity associated with a sparse elimination algorithm. The focus is on leveraging the "bordering algorithm" which minimizes storage requirements for pointers and row/column indices compared to traditional sparse elimination implementations. This reduction in storage is achieved through the use of the m-tree, a particular spanning tree of the graph of the filled-in matrix.

Sparse Elimination and m-Trees

The authors underscore the significance of m-trees in the context of numerical factorization of sparse matrices, although their application in this specific function appears novel in the existing body of work. The m-tree has, however, been utilized either directly or indirectly in optimal order algorithms aimed at computing fill-ins during symbolic factorization phases. This is notably discussed within the context of existing literature that includes contributions from Eisenstat et al., George and Liu, and others.

Methodological Details

Assuming that the sparse matrix $A$ has been preordered appropriately, the paper bypasses issues related to the choice of ordering algorithm, such as nested dissection or minimum degree, instead concentrating on the algorithmic complexity. Notably, the multigrid coarsening method is adapted to tackle anisotropic problems, employing plane relaxation to achieve effective smoothing factors when extending the methodology to three-dimensional cases.

The comparative analysis highlights differences in complexity between former and novel approaches to intersection problems for grids ordered by nested dissection. The transition from a cubic complexity $O(n^3)$ using traditional methods to $O(n^2 (\log n)^2)$ with the modified approach demonstrates the significance of the proposed methodology. This efficiency gain holds vast potential for numerical computation applications, particularly concerning extensive grid systems.

Practical and Theoretical Implications

From a theoretical perspective, the paper of m-trees and their integration into the sparse elimination algorithm provides a notable advancement in the understanding of nonnumerical complexities involved. Practically, the reduction in computational complexity signifies enhanced performance potential for software implementations that demand efficient data storage and computational processing of sparse matrices. As computational capabilities and requirements evolve, the insights within this paper contribute vital groundwork for future developments in algorithm design, particularly for large-scale simulations demanding sparse matrix operations.

The future trajectory of AI largely hinges on computational optimizations like those discussed in this paper. Continued research could explore even more efficient orderings and tree-based structures, potentially offering further reductions in operational complexities for increasingly intricate computational tasks. Moreover, the integration of such methods with advanced machine learning algorithms could lead to significant strides in processing efficiency and capability across numerous applications.