Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems (1801.01236v1)

Abstract: The process of transforming observed data into predictive mathematical models of the physical world has always been paramount in science and engineering. Although data is currently being collected at an ever-increasing pace, devising meaningful models out of such observations in an automated fashion still remains an open problem. In this work, we put forth a machine learning approach for identifying nonlinear dynamical systems from data. Specifically, we blend classical tools from numerical analysis, namely the multi-step time-stepping schemes, with powerful nonlinear function approximators, namely deep neural networks, to distill the mechanisms that govern the evolution of a given data-set. We test the effectiveness of our approach for several benchmark problems involving the identification of complex, nonlinear and chaotic dynamics, and we demonstrate how this allows us to accurately learn the dynamics, forecast future states, and identify basins of attraction. In particular, we study the Lorenz system, the fluid flow behind a cylinder, the Hopf bifurcation, and the Glycoltic oscillator model as an example of complicated nonlinear dynamics typical of biological systems.

• The paper introduces a novel method integrating multistep numerical schemes with deep neural networks to model complex nonlinear dynamics.
• The paper demonstrates high accuracy and robustness across systems like the Lorenz attractor, fluid flows, and biological oscillations.
• The paper highlights the approach's flexibility in handling noise, varying step sizes, and potential for future research in irregular data and enhanced architectures.

Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems

The paper "Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems" contributes to the field of systems identification by proposing a novel approach that synergizes classical numerical analysis techniques with contemporary deep learning methodologies. The primary objective is to identify complex nonlinear dynamical systems directly from data, thereby circumventing the limitations faced by traditional first principles modeling when dealing with highly complex systems.

Methodology

The core proposition of the paper is to integrate multistep time-stepping schemes from numerical analysis with deep neural networks to model nonlinear dynamical systems. The approach involves applying a linear multistep method to discretize the temporal dynamics and then leveraging deep neural networks to approximate the nonlinear function governing these dynamics. This framework is designed to handle scenarios where direct access to temporal gradients is infeasible and eliminates the need for a predefined set of basis functions.

The methodology presents significant flexibility by allowing for various multistep schemes like Adams-Bashforth, Adams-Moulton, and BDF methods. A neural network is trained to minimize the mean squared error between predicted and observed data, effectively learning the dynamics of the system.

Empirical Evaluation

The paper evaluates the effectiveness of the proposed approach on benchmark problems, including the identification of the Lorenz system, fluid flow past a cylinder described by the Navier-Stokes equations, the Hopf bifurcation model, and the glycolytic oscillator model.

• Performance Metrics: One of the strengths highlighted is the relative accuracy achieved across these diverse systems. Different multistep schemes were compared, and within this context, the Adams-Moulton scheme consistently resulted in higher accuracy.
• Robustness to Noise and Step Size: The paper reports performance metrics under varying temporal gaps and noise levels, revealing the robustness of the approach. In some instances, larger temporal gaps or noise levels led to improved accuracy, suggesting a natural robustness to certain perturbations.
• Neural Network Flexibility: The approach demonstrates adaptability across different neural network architectures, identifying that deeper networks often yield better performance. However, the width of these networks has an unexpected impact, with more neurons occasionally degrading performance.

Implications and Future Directions

The theoretical implications of this research span both practical and academic realms of AI and nonlinear dynamics. The proposed methodology provides a scalable framework for extracting dynamic models that contribute to a deeper understanding of complex systems, such as fluid mechanics and biological oscillations, without pre-assumed dynamics.

Practically, this advancement allows researchers to construct predictive models based on data-driven insights where traditional methods are rendered infeasible due to complexity or lack of complete system understanding. The integration of multistep schemes provides a natural way of handling memory effects within temporal data, offering an advantage over some existing approaches that rely on fully Markovian assumptions or require more complex architectures such as RNNs.

Looking forward, the paper identifies several open research areas including handling irregularly sampled data, enhancing the model with regularization techniques, and incorporating partial knowledge of the system dynamics for more informed modeling. Additionally, potential exploration into convolutional architectures could address high-dimensional input complexities, enhancing the approach's applicability to broader areas such as image-based flows and spatio-temporal dynamics.

Conclusion

Overall, the work provides a substantial contribution to the body of knowledge in systems identification leveraging machine learning. While the paper presents promising initial results, it lays the groundwork for extensive further research into more robust, interpretable, and scalable methods for understanding complex nonlinear systems through data-driven approaches.