Learning Dissipative Dynamics in Chaotic Systems (2106.06898v2)

Abstract: Chaotic systems are notoriously challenging to predict because of their sensitivity to perturbations and errors due to time stepping. Despite this unpredictable behavior, for many dissipative systems the statistics of the long term trajectories are governed by an invariant measure supported on a set, known as the global attractor; for many problems this set is finite dimensional, even if the state space is infinite dimensional. For Markovian systems, the statistical properties of long-term trajectories are uniquely determined by the solution operator that maps the evolution of the system over arbitrary positive time increments. In this work, we propose a machine learning framework to learn the underlying solution operator for dissipative chaotic systems, showing that the resulting learned operator accurately captures short-time trajectories and long-time statistical behavior. Using this framework, we are able to predict various statistics of the invariant measure for the turbulent Kolmogorov Flow dynamics with Reynolds numbers up to 5000.

• The paper introduces the Markov Neural Operator to approximate solution operators for chaotic dissipative systems.
• It employs dissipativity regularization and Sobolev loss to capture both low- and high-frequency dynamics effectively.
• Empirical results on PDEs, including the Kuramoto-Sivashinsky and Navier-Stokes equations, demonstrate reliable long-term statistical predictions.

Learning Dissipative Dynamics in Chaotic Systems

This paper addresses the challenge of predicting the long-term behavior of chaotic dynamical systems using machine learning techniques, specifically focusing on systems with dissipative properties. Such systems, despite their initial unpredictability due to sensitivity to perturbations, often display stable statistical properties over the long term. Notably, these properties are described by an invariant measure supported on a set called the global attractor. The paper presents a machine learning framework to learn solution operators for these systems, highlighting both methodological innovations and empirical results.

The authors propose the Markov Neural Operator (MNO), a neural operator-based method designed to approximate the solution operator of chaotic systems. Neural operators are specifically tailored to map between function spaces, allowing them to handle the infinite dimensions often associated with dissipative systems described by partial differential equations (PDEs). This approach leverages the ergodic nature of many chaotic systems, enabling long-term statistical predictions without the need for precise trajectory predictions over extended time horizons.

In validating their approach, the authors demonstrate the capability of the MNO to learn and predict statistical characteristics such as Fourier spectra, auto-correlation, and energy spectra from chaotic systems including the Kuramoto-Sivashinsky equation and the 2D Navier-Stokes equation in the form of Kolmogorov flows. For these systems, the solution operator was approximated effectively, capturing both short-time trajectories and long-term statistical behavior even at high Reynolds numbers, approaching 5000.

The paper emphasizes several critical components of their methodology:

1. Dissipativity Regularization: Empirically shown to stabilize long-term predictions by including a regularization term enforcing dissipative dynamics. This not only prevents blow-ups and collapses in predicted trajectories but also ensures the model adheres to the dissipative properties observed in practical systems.
2. Sobolev Loss: The incorporation of Sobolev norms in training enhances the model’s ability to capture high-frequency details and derivative information, crucial for maintaining accuracy in function space models.
3. Time-Step Selection: Investigations into the optimal choice of time steps for training reveal a complex interplay between per-step error and long-term prediction accuracy. The authors demonstrate that training with appropriately chosen time steps is essential for accurate long-term dynamics modeling.
4. Theoretical Guarantees: The paper furnishes theoretical proofs guaranteeing that, under appropriate conditions, MNOs can approximate the solution operator of infinite-dimensional systems to an arbitrary degree of accuracy, thus substantiating its applicability to a broad class of chaotic systems.

The implications of this work extend across both theoretical and practical domains. Theoretically, it sharpens the understanding of how machine learning models can replicate and predict complex dynamics governed by PDEs. Practically, it opens avenues for applying such techniques in fields where simulating chaotic systems is essential, such as climate modeling and fluid dynamics.

For future work, the authors suggest exploring applications to systems with significant path-dependence or exploring non-ergodic systems. Additionally, formalizing error bounds for infinite time horizons remains a crucial step toward establishing rigorous reliability for prediction in chaotic systems.

In conclusion, the authors successfully present a machine learning framework that fundamentally respects the structure of chaotic dynamics, offering both rigorous theoretical foundations and strong empirical performance in learning dissipative chaotic systems. Their approach enhances the potential of machine learning in scientific computing by combining principled methods from dynamical systems theory with advanced neural architectures.