Deep learning for universal linear embeddings of nonlinear dynamics (1712.09707v2)

Abstract: Identifying coordinate transformations that make strongly nonlinear dynamics approximately linear is a central challenge in modern dynamical systems. These transformations have the potential to enable prediction, estimation, and control of nonlinear systems using standard linear theory. The Koopman operator has emerged as a leading data-driven embedding, as eigenfunctions of this operator provide intrinsic coordinates that globally linearize the dynamics. However, identifying and representing these eigenfunctions has proven to be mathematically and computationally challenging. This work leverages the power of deep learning to discover representations of Koopman eigenfunctions from trajectory data of dynamical systems. Our network is parsimonious and interpretable by construction, embedding the dynamics on a low-dimensional manifold that is of the intrinsic rank of the dynamics and parameterized by the Koopman eigenfunctions. In particular, we identify nonlinear coordinates on which the dynamics are globally linear using a modified auto-encoder. We also generalize Koopman representations to include a ubiquitous class of systems that exhibit continuous spectra, ranging from the simple pendulum to nonlinear optics and broadband turbulence. Our framework parametrizes the continuous frequency using an auxiliary network, enabling a compact and efficient embedding at the intrinsic rank, while connecting our models to half a century of asymptotics. In this way, we benefit from the power and generality of deep learning, while retaining the physical interpretability of Koopman embeddings.

• The paper introduces a deep auto-encoder that discovers intrinsic coordinates to linearize complex nonlinear dynamics.
• The method generalizes Koopman representations for systems with continuous spectra, enhancing interpretability and efficiency.
• Numerical experiments demonstrate effective linearization in models ranging from nonlinear pendulums to high-dimensional fluid flows.

Deep Learning for Universal Linear Embeddings of Nonlinear Dynamics

The paper "Deep Learning for Universal Linear Embeddings of Nonlinear Dynamics" by Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton presents a novel approach to identify coordinate transformations that linearize strongly nonlinear dynamic systems. The paper leverages deep learning to discover and represent Koopman eigenfunctions from trajectory data, embedding the nonlinear dynamics into low-dimensional, interpretable linear frameworks.

Key Contributions

The research introduces a modified deep auto-encoder network to identify and parameterize nonlinear coordinates on which the dynamics are globally linear. The following salient features highlight the core contributions of the paper:

1. Parsimony and Interpretability: The proposed network embeds dynamics on a low-dimensional manifold parameterized by Koopman eigenfunctions. The architecture is designed to be parsimonious and interpretable by construction.
2. Extension to Continuous Spectra Systems: The framework generalizes Koopman representations to systems with continuous spectra, common in various physical phenomena such as nonlinear optics and turbulence. By using an auxiliary network to parametrize the continuous frequency, a compact and efficient embedding is achieved.
3. Effective Use of Deep Learning: The power of deep learning is harnessed to balance between flexible data representation and retaining physical interpretability inherent in Koopman embeddings.

Methodological Framework

The network architecture comprises an auto-encoder augmented by loss functions to enforce the linearity of the dynamics in the intrinsic coordinates. The primary innovations presented include:

• Auto-Encoder Structure: The auto-encoder identifies intrinsic coordinates on which the dynamics evolve linearly. The encoder maps the state of the system to these coordinates, while the decoder reconstructs the state from the coordinates.
• Auxiliary Network for Continuous Spectrum: For systems with continuous spectra, an auxiliary network determines the parametric dependency of the Koopman operator on the frequency, allowing the intrinsic coordinates to be mapped into nearly perfect sinusoidal functions.
• Multi-Component Loss Function: The training process involves a composite loss function comprising reconstruction accuracy, prediction accuracy, and linearity of the dynamics in the intrinsic coordinates.

Numerical Results

The effectiveness of the proposed deep learning approach is demonstrated through applications to several dynamical systems, highlighting various forms of eigenvalue spectra and state dimensionalities:

1. Simple Model with Discrete Spectrum: The method accurately recovers the Koopman eigenfunctions for a low-dimensional system with a discrete spectrum, demonstrating the network's ability to linearize the dynamics faithfully.
2. Nonlinear Pendulum with Continuous Spectrum: The framework channels the continuous eigenvalue spectrum into an interpretable model. By capturing the continuous frequency variation a function of intrinsic coordinates, the approach provides a parsimonious representation without resorting to high-order asymptotic expansions.
3. High-Dimensional Fluid Flow: The method is applied to the classic problem of fluid flow past a circular cylinder at Reynolds number 100, demonstrating its capacity to manage high-dimensional data and capture the essential dynamics with minimal eigenfunctions.

Implications and Future Work

The implications of this research are both practical and theoretical. Practically, the approach presents a scalable and interpretable architecture to linearize nonlinear dynamical systems, potentially benefiting areas like climate modeling, neuroscience, and complex network analysis where data-driven models are crucial. Theoretically, the integration of deep learning with Koopman theory opens avenues for discovering new symmetries and conservation laws in complex systems.

Future work could explore extending these techniques to more complex turbulent flows, advancing nonlinear control algorithms, and applying them to fields such as epidemiology and finance. Moreover, the inclusion of physical constraints directly into the network architecture could further enhance model accuracy and interpretability.

Conclusion

The deep learning approach for universal linear embeddings of nonlinear dynamics presents a significant step towards making powerful linear analysis tools accessible for complex systems. By embedding the nonlinear dynamics into low-dimensional, interpretable linear frameworks, the method provides both theoretical and practical advancements, bridging gaps between modern deep learning techniques and classical dynamical systems theory.