Dictionary learning for Kernel EDMD

Abstract: Studying nonlinear dynamical systems through their state space behavior can be challenging, and one possible alternative is to analyze them via their associated Koopman operator. This turns the nonlinear problem into a linear, infinite-dimensional one. To approximate the operator in finite dimensions, extended dynamic mode decomposition (EDMD) is a commonly used algorithm. It requires a finite list of functionals and a set of snapshots from the system to compute an approximation of the operator and its corresponding spectrum. Instead of choosing the list of functionals directly, it can be implicitly defined via kernels, a method known as kernel extended dynamic mode decomposition (kEDMD). However, one still needs to define the kernel and choose its parameter values. In this paper, we aim to streamline this process by extending dictionary learning for EDMD to kernel learning in kEDMD. By simplifying kEDMD we show how to perform gradient-based optimization over the learnable kernel parameters, and demonstrate that this method leads to useful kernels for the original kEDMD. The focus of our work is a method that takes a weighted list of kernels with randomly initialized values as input and outputs a list of kernels and parameter values suitable for approximating the Koopman operator of the underlying system. We demonstrate that unimportant kernels can be removed from the list by analyzing the weights in the weighted sum. We evaluate the method across several experiments, including the Duffing oscillator and the Kuramoto-Sivashinsky PDE, showcasing the method's different strengths.

• The paper formalizes dictionary learning in the kEDMD framework through rigorous optimization, establishing equivalences with truncated-SVD approaches.
• It introduces the simplified skEDMD variant that leverages explicit kernel evaluations and L1 regularization to enhance interpretability and computational efficiency.
• Empirical evaluations on systems like the Duffing oscillator and KS PDE demonstrate improved trajectory prediction and effective kernel weight selection.

Dictionary Learning for Kernel EDMD: A Rigorous Study

Introduction and Theoretical Framework

This work rigorously addresses kernel selection in the kernel extended dynamic mode decomposition (kEDMD) method for approximating the Koopman operator, focusing on systematic dictionary (i.e., observable set or kernel) learning as an explicit optimization problem. The Koopman operator provides a linear (though infinite-dimensional) perspective for analyzing nonlinear dynamical systems. In practice, one must work with finite-dimensional approximations. The canonical EDMD constructs such approximations by projecting dynamics onto a finite dictionary of observables; kernel-based variants (kEDMD) define this dictionary implicitly via kernel functions, enabling implicit large or infinite-dimensional lifted spaces with a compact matrix representation.

The main challenge, both theoretical and practical, is that the efficacy of Koopman-based prediction and control relies critically on the choice of dictionary (or kernel). Historically, such choices have been made heuristically and require intensive manual tuning, especially in kernel-based approaches where the dictionary is implicit and parameter choices can be highly nontrivial. This paper formalizes dictionary learning in the kernel EDMD setting—specifically, it elaborates both a direct (but computationally limiting) approach for dictionary learning in the implicit kEDMD framework and, crucially, develops and justifies a simplified, computationally efficient variant (skEDMD) that is provably equivalent to traditional truncated-SVD-based kernel EDMD (kEDMD).

Methodological Contributions

The core technical contributions span several intertwined elements:

• Loss function construction for kernel learning: The authors systematically derive loss functions for gradient-based kernel parameter optimization, covering prediction, dictionary, and eigenfunction-based losses. They show theoretically how these can, in principle, be computed in standard kEDMD (with associated restrictions), but that a simplified approach circumvents computational and practical obstacles.
• Simplified skEDMD variant: Through precise linear algebraic arguments, skEDMD is shown to be equivalent to truncated kEDMD, with an explicit mapping between their Koopman matrix representations and eigendecompositions. The skEDMD formulation directly constructs the dictionary as kernel evaluations against training data, allowing EDMD-type computations and pointwise evaluation of approximate eigenfunctions.
• Weighted kernel sums and sparsification: Kernels are parameterized as weighted sums of basic kernels (e.g., RBFs, periodic, linear, neural network), with outer weights subject to L1 regularization for sparsity. This facilitates both interpretability and automatic kernel selection. Uninformative kernels are pruned adaptively via weight thresholds, in direct analogy to ideas from automatic relevance determination.
• Algorithmic modernities: The implementation leverages SGD on mini-batches, adaptive subsampling for computational efficiency, and regularization schedulers to stabilize training, particularly for ill-conditioned or poorly-initialized kernels.

Empirical Evaluation

One-Parameter Kernel Test Case: Duffing Oscillator

A single-parameter RBF kernel is learned for the Duffing oscillator. The importance of loss function selection is highlighted. Prediction losses—unlike raw dictionary or eigenfunction-based losses—drive meaningful convergence and rapid improvement in trajectory and eigenvalue predictions.

Figure 1: Different loss functions over training epochs for the Duffing oscillator, demonstrating only prediction-type losses reliably decrease with training.

Figure 2: Test set trajectories show that post-training kernels capture true dynamics while initial (mis-scaled) kernels do not.

Figure 3: Pre- and post-training Koopman spectrum. Training drives proliferation of nontrivial eigenvalues necessary for accurate trajectory modeling.

Weighted Kernel Identification: Synthetic Modulo System

A synthetic system with abrupt phase-wrapped dynamics is used to demonstrate kernel relevance selection. The learned kernel weights strongly emphasize the kernel that encodes appropriate circular topology (via $\cos$ and $\sin$ embedding), essentially disregarding linear and standard RBF kernels, which are inherently incompatible with the underlying dynamics.

Figure 4: Learned vs. initial Koopman spectra with varied kernel weights—a post-training spectrum matches analytically predicted eigenvalues, providing evidence of correct kernel identification and selection.

Complex PDE Example: Kuramoto-Sivashinsky Equation

A high-dimensional, spatiotemporal benchmark—KS PDE—is used to assess the approach under challenging conditions, with a composite kernel comprising multiple RBFs, cosine, linear, and a nonparametric (NNGP) kernel. Pruning, re-initialization, and fine-tuning enable both efficient representation and improved approximation of eigenstructure and trajectories. Instabilities observed for poorly-initialized or overly flexible kernels are effectively controlled via regularization scheduling.

Figure 5: Loss curves during KS PDE training, showing loss alignment across training and validation, and stability improvements via kernel learning.

Figure 6: Representative test trajectories showing improved match to ground truth after training composite and pruned kernels.

Figure 7: Koopman spectrum with color-coded residuals before and after training. Post-training, eigenvalues with lower residuals cluster more tightly inside the unit circle, corresponding to meaningful dynamical modes.

Figure 8: Test KS trajectories post-pruning. Kernel learning and sparsification continue to yield high-quality predictions.

Practical Implications and Theoretical Impact

The formalization and practical implementation of dictionary learning for kEDMD via the skEDMD approach constitute a substantial advance toward fully automating operator-theoretic learning for nonlinear dynamics. Specific technical strengths include:

• End-to-end kernel selection: No manual parameter studies are needed—a randomized, overparameterized kernel family is provided, and the method prunes to compact, data-driven, relevant structures.
• Computational tractability: Equivalence to truncated SVD approaches, with all operations remaining within numerically accessible domains (i.e., kernel matrices with size determined by snapshot count, not implicit dictionary size), makes large-scale application feasible.
• Broad extensibility: The framework may be adapted to incorporate richer kernel parameterizations (including neural kernels and nonparametric flows), advanced regularization strategies, or even resDMD-based spectral pollution filtering as part of the optimization loop, as discussed in directions for future research.

Theoretically, the work ensures that all technical mappings (e.g., eigenvector/eigenfunction correspondences between representations) are explicit and tractable, ensuring operator–kernel relationships are maintained throughout optimization.

Limitations and Future Research

Though robust for a wide class of smooth and moderately stiff kernels, some limitations are noted with nonsmooth kernels (e.g., Matérn family), where the training landscape can become challenging. The number of required snapshots (i.e., truncation size $\tilde N$) may become substantial in certain "hard" systems, and tuning hyperparameters—especially regularization schedules—still benefits from some user attention. Future work should clarify the most effective combination of loss terms, leverage recent advances in eigenpair filtering (e.g., resDMD), and expand expressive kernel classes with minimal human intervention.

Conclusion

This paper rigorously establishes dictionary learning for kEDMD in both theory and practice, providing a tractable, extensible, and interpretable approach to learning Koopman operator approximations via automatic kernel selection and parameterization. Empirical results across synthetic and high-dimensional nonlinear systems validate the effectiveness and versatility of the approach. This work creates a foundation for further developments toward automated, fully data-driven operator-theoretic learning in nonlinear dynamical systems.