Universal Delay Embedding (UDE)

• Universal Delay Embedding (UDE) is a framework that transforms raw time-series data into high-dimensional delay-coordinate spaces, preserving the system’s inherent topology.
• It integrates delay embedding theory with Koopman operator methods and deep learning (via patch tokenization) to enable scalable, interpretable forecasting of nonlinear dynamics.
• Empirical evaluations demonstrate that UDE achieves a 20–23% reduction in forecasting error and robust domain generalization even with limited labeled data.

Universal Delay Embedding (UDE) is a framework for time-series modeling, representation, and forecasting in which raw temporal data are systematically transformed into higher-dimensional spaces using delay-coordinate maps, leveraging the foundational guarantees of Takens’ embedding theorem. The concept generalizes and integrates delay embedding theory with Koopman operator methods and modern deep learning architectures to enable scalable, interpretable, and universal forecasting across a wide range of nonlinear dynamical systems and data domains (Wang et al., 15 Sep 2025). UDE approaches aim to preserve and exploit the underlying geometric, topological, and spectral properties of dynamical systems, allowing both the reconstruction and prediction of complex system trajectories in a data-driven, theoretically grounded manner.

1. Foundations of Delay Embedding and Takens’ Theorem

Takens’ embedding theorem forms the mathematical backbone of Universal Delay Embedding. The theorem asserts that, under generic conditions, the entire state of a smooth dynamical system evolving on a compact attractor of dimension $d$ can be reconstructed from time-delayed observations of a single scalar observable, provided the embedding dimension $m$ exceeds $2d$. The delay-coordinate vector is defined as

$y_t = [x_{t-(m-1)},\ x_{t-(m-2)},\ ..., x_t]$

where $\{x_t\}$ is the observed scalar time series. Arranging successive delay vectors yields a geometrical reconstruction of the system’s attractor that is homeomorphic to the original, thereby preserving both topological structure and qualitative dynamic evolution.

In practical UDE frameworks, these delay-embedded vectors are organized into Hankel matrices,

$$H = \begin{bmatrix} x_1 & x_2 & \cdots & x_{L} \ x_2 & x_3 & \cdots & x_{L+1} \ \vdots & \vdots & & \vdots \ x_m & x_{m+1} & \cdots & x_{L+m-1} \end{bmatrix}$$

from which substructures can be extracted for further analysis and modeling (Wang et al., 15 Sep 2025).

The preservation of the attractor’s topology in the embedded space provides a strong foundation for further symmetry-based, geometric, or operator-theoretic modeling.

2. Koopman Operator Formulation in UDE

A central innovation of Universal Delay Embedding lies in the application of Koopman operator theory. The Koopman operator is an infinite-dimensional linear operator that encodes the evolution of observables (functions on the state space) under the action of the nonlinear dynamics. Within UDE, this perspective enables nonlinear evolution in the original state space to be lifted and approximated by a finite-dimensional linear operator in a learned latent feature space.

Given a nonlinear transformation $f$ (e.g., a deep neural network encoder) that lifts each delay-embedded vector $y_t$ to a latent state $z_t = f(y_t)$, the discrete-time dynamical evolution is approximated by a Koopman operator $g$ such that

$z_{t+h} = g(z_t)$

where $g$ is linear in the latent space, despite the underlying system being nonlinear.

This architecture permits efficient prediction and analysis of nonlinear dynamics through linear evolution in feature space, an approach that provides interpretability and analytical tractability (Wang et al., 15 Sep 2025).

3. Delay-Embedded Patch Tokenization and Deep Learning Integration

UDE frameworks operationalize delay embedding by segmenting Hankel matrices into non-overlapping two-dimensional patches along both the time and delay axes. Each patch (a submatrix of size $p \times q$) encodes local geometric and temporal dynamics and is treated as a token, in analogy to image patches in vision transformers.

The sequence of such tokens is processed by a self-attention encoder—typically a Transformer architecture—with the following data flow:

• Flatten and linearly project each patch to a token embedding.
• Add fixed sinusoidal positional encodings to preserve sequential and spatial relationships.
• Apply several layers of multi-head self-attention and feed-forward blocks, enabling global context aggregation while attending to local dynamical structure.
• Optionally employ token pooling to reduce sequence length and computational complexity.

The result is a set of latent representations in which predictions are made by applying the learned Koopman operator’s linear action, followed by a reconstruction decoder to map latent states back to observable time-series predictions (Wang et al., 15 Sep 2025). This structure generalizes across univariate and multivariate series by independently (but with parameter sharing) processing each channel, supporting scalability.

4. Theoretical Guarantees and Interpretability

The UDE methodology is grounded in rigorous theoretical guarantees:

• Topological invariance: By Takens’ theorem, the attractor’s homeomorphism is preserved in the delay-embedded space if the embedding dimension is sufficiently large.
• Local geometry preservation: The patch extraction process ensures that each token retains localized geometric and dynamic information, with persistent homology analyses confirming that topological features (e.g., loops or holes in phase portraits) are preserved within and across patches.
• Spectral and operator-theoretic equivalence: Koopman operator approximations in the latent space inherit spectral invariants from the original dynamics, enabling the model to recover domain-invariant dynamical features (Susuki et al., 2017).

Model attention visualizations show that only a sparse subset of tokens—termed "dynamical anchors" (Editor's term)—consistently attract high attention, further supporting interpretability in terms of physical or dynamical modes.

5. Practical Application and Empirical Performance

UDE is designed as a foundation model for universal time-series forecasting. Empirical evaluations showcase:

• Zero-shot generalization: UDE pretrained on large-scale datasets (e.g., USTD-12B) achieves a 20–23% average reduction in mean squared error for forecasting tasks relative to state-of-the-art baselines—including Timer-16B and Moirai-M—without additional training (Wang et al., 15 Sep 2025).
• Data-efficient fine-tuning: When labeled data are scarce (e.g., only 10–20% available), UDE consistently achieves superior predictive accuracy and adapts rapidly.
• Domain generalization: Across diverse benchmarks, including real-world climate (ERA5) and city-level meteorological variables, UDE provides robust forecasts, outperforming conventional and contemporary models.
• Interpretability: Token clustering in the learned feature space, measured by 2-Wasserstein distances between patch-wise persistence diagrams, reveals that similar dynamical modes group together, allowing domain-informed analysis of system evolution.

These empirical features demonstrate UDE’s cross-domain scalability and robustness, and its interpretability in extracting physically meaningful patterns.

6. Comparison with Previous Methods

UDE extends and unifies several prior strands of research:

• Traditional delay embedding methods (e.g., Takens/Sauer–Yorke–Casdagli) focused on state-space reconstruction for low-dimensional nonlinear systems but did not integrate advanced learning or operator-theoretic approaches, nor did they address large-scale, foundation-model ambitions.
• Koopman operator learning provided linearizations of nonlinear systems but historically required explicit basis construction, limited scalability, and lacked principled delay-embedding integration.
• Self-attention models for time series have excelled in representation learning but, without delay embedding, often lack theoretical guarantees concerning attractor topology or interpretability.
• Combined operator-theoretic deep learning models (e.g., Deep Koopman, TimeVQVAE) only partially integrate insight from Takens’ theorem, whereas UDE retains theoretical guarantees at every stage.

By contrast, UDE’s delay embedding foundation, patchwise tokenization, and explicit Koopman operator mapping in deep latent spaces accomplish both universal applicability and interpretability (Wang et al., 15 Sep 2025).

7. Implications and Future Directions

UDE establishes a scalable, interpretable paradigm for universal time-series modeling and forecasting, with broad potential for scientific and industrial applications (e.g., climate dynamics, chaotic systems, financial forecasting, infrastructure monitoring).

Possible extensions include:

• Dynamic data-driven adjustment of embedding and patch parameters via topological persistence analyses.
• Integration with vision-LLMs for spatiotemporal forecasting informed by multimodal (e.g., satellite, sensor, textual) data.
• Enhanced anomaly detection and robustness to extreme events through the explicit tracking of patchwise topological features.

A plausible implication is that, as both data availability and computational resources expand, UDE-style paradigms grounded in universal embedding principles will become the standard baseline for time-series analysis in domains requiring both predictive accuracy and physical interpretability.

---

Table: Key Components of UDE in Comparison to Classical Approaches

Component	Classical Delay Embedding	Universal Delay Embedding (UDE)
Embedding Mechanism	Delay vectors via Takens’ theorem	Delay vectors + Hankel matrix + patching
Operator Perspective	Implicit (reconstruction only)	Explicit (Koopman operator in latent space)
Modeling Infrastructure	Statistical or geometric analysis	Transformer-based self-attention encoder
Topology/Geometry	Homeomorphism (theorem guarantee)	Persistent homology, token-wise topology
Predictive Workflow	Local (nearest neighbor, SVD, etc)	Linear latent-space prediction (Koopman)
Interpretability	Phase-space plots, Lyapunov exponents	Token clustering, cross-layer attention
Scalability	Limited by embedding, no patching	Foundation model, modular, patch-wise

---

Universal Delay Embedding offers a theoretically rigorous and empirically validated methodology for representing, analyzing, and forecasting time-series data in nonlinear dynamical systems, enabling both high accuracy and interpretability in large-scale, cross-domain scenarios (Wang et al., 15 Sep 2025).