CKNet: A Convolutional Neural Network Based on Koopman Operator for Modeling Latent Dynamics from Pixels (2102.10205v2)

Abstract: With the development of end-to-end control based on deep learning, it is important to study new system modeling techniques to realize dynamics modeling with high-dimensional inputs. In this paper, a novel Koopman-based deep convolutional network, called CKNet, is proposed to identify latent dynamics from raw pixels. CKNet learns an encoder and decoder to play the role of the Koopman eigenfunctions and modes, respectively. The Koopman eigenvalues can be approximated by eigenvalues of the learned state transition matrix. The deterministic convolutional Koopman network (DCKNet) and the variational convolutional Koopman network (VCKNet) are proposed to span some subspace for approximating the Koopman operator respectively. Because CKNet is trained under the constraints of the Koopman theory, the identified latent dynamics is in a linear form and has good interpretability. Besides, the state transition and control matrices are trained as trainable tensors so that the identified dynamics is also time-invariant. We also design an auxiliary weight term for reducing multi-step linearity and prediction losses. Experiments were conducted on two offline trained and four online trained nonlinear forced dynamical systems with continuous action spaces in Gym and Mujoco environment respectively, and the results show that identified dynamics are adequate for approximating the latent dynamics and generating clear images. Especially for offline trained cases, this work confirms CKNet from a novel perspective that we visualize the evolutionary processes of the latent states and the Koopman eigenfunctions with DCKNet and VCKNet separately to each task based on the same episode and results demonstrate that different approaches learn similar features in shapes.

• The paper introduces CKNet, a novel deep convolutional network using Koopman operator theory to model latent dynamics from high-dimensional pixel data.
• CKNet integrates encoder-decoder CNNs with deterministic and variational convolutional Koopman networks to encode pixels into linear latent representations, overcoming challenges of manual feature design.
• Experimental validation on OpenAI Gym and MuJoCo tasks demonstrates CKNet's robustness and accuracy in capturing system dynamics and forecasting future states.

A Koopman-based Deep Convolutional Network for Modeling Latent Dynamics from Pixels

This paper presents a novel approach to dynamical system modeling using a Koopman-based deep convolutional network, CKNet. The research addresses the challenge of modeling high-dimensional systems from pixel-level observations—specifically, focusing on latent dynamics identification. The CKNet framework leverages the Koopman operator theory, which offers a linear perspective on non-linear dynamic systems, facilitating the linear approximation of complex systems. CKNet integrates this theoretical foundation with deep learning capabilities, particularly convolutional neural networks (CNNs), to map high-dimensional pixel inputs to latent states that capture the intrinsic dynamics of a system.

The core of CKNet involves the synthesis of an encoder-decoder architecture with the deterministic convolutional Koopman network (DCKNet) and the variational convolutional Koopman network (VCKNet). These networks are designed to encode raw pixels into latent representations—acting respectively as eigenfunctions of the Koopman operator—and then decode these representations back to observable states. The Koopman eigenvalues, critical to this approach, are derived from the learned state transition matrix, transforming the problem of non-linear dynamics into a tractable linear form.

Methodology

CKNet is applied in two paradigms: unforced and forced systems. For unforced systems, the encoder generates features that span a finite-dimensional subspace, approximating the infinite-dimensional function space where the Koopman operator acts linearly. For forced systems, additional control inputs are incorporated into the dynamical model, enabling the approximation of systems that respond to external influences.

The framework applies the extended dynamic mode decomposition (EDMD) in conjunction with deep neural networks for modeling, using CNNs to replace manually designed kernel functions, a strategy that aids in coping with large-scale input dimensions, such as raw image pixels.

Key innovations of CKNet include:

1. Dynamic Mode Decomposition (DMD) Processing: While previous DMD methods relied on singular value decomposition (SVD) and kernel-based functions, CKNet autonomously learns appropriate dictionary functions through a deep neural network, circumventing the complexities of manual dictionary design for high-dimensional data.
2. Auxiliary Weight Term: The research introduces an auxiliary weight in the loss function framework, designed to enhance the persistence and accuracy of multi-step predictions, crucial for modeling systems with long-term dependencies.
3. Controllability and Interpretability: The linear nature of the identified dynamics posits an ability for controller design—an advantage in engineering fields where control synthesis is key.

Experimental Validation

CKNet's potential is validated through simulations across several environments: the OpenAI Gym, which includes CartPole and MountainCar tasks, and MuJoCo, with a selection of control tasks such as Cheetah-run and Walker-walk. These experiments highlight the robustness of CKNet in capturing the dynamics and forecasting future states with high accuracy, both offline and online.

Implications and Future Directions

This research contributes to the broader field of system dynamics and control by demonstrating a bridge between data-driven deep learning techniques and classical theoretical frameworks like the Koopman operator. Practically, CKNet could revolutionize applications where real-time modeling of nonlinear systems is essential, such as autonomous driving and robotic control.

Future work could focus on expanding CKNet's applicability to even more complex systems and further integration with reinforcement learning paradigms. There is also room for exploration into how these latent state spaces could enhance transfer learning or domain adaptation across different system architectures. Overall, CKNet lays down important groundwork for future exploration of Koopman theory in high-dimensional latent space modeling.