Derivative Delay Embedding (DDE) Overview

• Derivative Delay Embedding (DDE) is a method that converts raw time series into a structured embedding space using finite differences and delay stacking.
• It employs a discretized grid-mapping approach to maintain fixed memory usage and operate in constant time for online, real-time scenarios.
• The Markov Geographic Model (MGM) augments DDE through probabilistic classification by combining geographic state distributions with transition statistics for enhanced noise robustness.

Derivative Delay Embedding (DDE) provides a principled approach for online modeling and classification of streaming time series, characterized by invariance to input length, phase, and baseline, and by computational efficiency suitable for real-time settings. Unlike classical fixed-length or batch-oriented techniques, DDE incrementally transforms raw time-series into a structured embedding space using finite differences and delay embeddings, enabling memory-efficient modeling regardless of stream length or alignment. The Markov Geographic Model (MGM) augments DDE with a nonparametric mechanism for probabilistic classification by leveraging both steady-state distribution and transition statistics in the discretized embedding space.

1. Mathematical Formulation of Derivative Delay Embedding

DDE operates on a discrete time-series $y_t \in \mathbb{R}$ (or $\mathbb{R}^n$), leveraging a finite difference to estimate derivatives: $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$ where $\tau$ is a positive integer lag (often $\tau=1$ for maximal temporal resolution). In practice, for each timestep $t$, the derivative signal is: $y'_t = (y_t - y_{t-\tau}) / \tau$ To form the DDE vector, stack $d$ such derivatives spaced by delay step $s$: $$\mathbf{v}_t = [y'_t, \; y'_{t-s},\; y'_{t-2s}, \; \ldots, \; y'_{t-(d-1)s}]^T \in \mathbb{R}^d$$ This vector retains latent dynamical information in the time series via recursive patterns.

To enable fixed memory usage, a $\mathbb{R}^n$0-dimensional grid overlays the continuous embedding space. The grid-mapping

$\mathbb{R}^n$1

rounds each coordinate of $\mathbb{R}^n$2 to its nearest cell index. The resulting discrete DDE state is: $\mathbb{R}^n$3 Empirical evidence suggests $\mathbb{R}^n$4 bins per axis provides adequate resolution and tractable statistics.

2. Online Computation and Streaming Efficiency

The incremental nature of DDE is realized by maintaining a circular buffer of length $\mathbb{R}^n$5 and updating as new data arrives:

• Upon receiving $\mathbb{R}^n$6, push to buffer; discard oldest sample.
• Compute $\mathbb{R}^n$7 via finite difference.
• Construct $\mathbb{R}^n$8 by gathering the required delayed derivatives.
• Discretize via grid-mapping $\mathbb{R}^n$9 to obtain $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$0.

Each update requires only $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$1 time and memory (plus a lightweight lookup for cell indexing), without regard for the total stream length. Thus, DDE is suitable for online, real-time systems and continuous data streams. This property sharply distinguishes DDE from batch/segmentation-based approaches requiring repeated normalization or windowing.

3. Theoretical Properties: Invariance and Embedding

Takens' theorem provides the foundational basis: delay embedding of a generic observable produces a diffeomorphic reconstruction of the latent dynamics when $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$2. DDE inherits and strengthens this property by operating on derivatives rather than raw signals, leading to several invariance properties:

• Additive shift invariance: Finite differences remove constant baseline drift, making the model robust to offset.
• Phase and length invariance: The embedding’s geography and attractor occupancy are determined by intrinsic, recurrent structure, not by stream alignment or segment length.
• Memory efficiency in infinite streams: Once the attractor region is reached, only a bounded finite set of grid-cells are revisited, allowing memory use to remain constant in unbounded streaming scenarios.

Replacing the observation function $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$3 in Takens' framework with the derivative $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$4 is theoretically justified, with $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$5 acting as a generic observable.

4. Markov Geographic Model (MGM) for Nonparametric Classification

MGM augments DDE for online, probabilistic classification by maintaining:

• Geographic state distribution: For each class $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$6, the frequency count $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$7 for each cell $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$8 traversed by the training trajectory is aggregated. The normalized log-scaled probability for cell $\dot{y}(t) \approx \frac{y_t - y_{t-\tau}}{\tau}$9 is:

$\tau$0

The logarithmic form softens the peak dominance of high-count cells typical near zero-crossings.

• Transition counts: MGM maintains a sparse count $\tau$1 for jumps from cell $\tau$2 (at $\tau$3) to $\tau$4 (at $\tau$5), converting to conditional probabilities:

$\tau$6

Test trajectories $\tau$7 are then scored by cumulative sum and product: $\tau$8 Online updates maintain running statistics, $\tau$9 per timestep, enabling immediate classification via $\tau=1$0.

5. Robustness Enhancement via Neighborhood Matching

Exact cell transitions may prove brittle due to noise or quantization artifacts. Robust classification incorporates local spatial neighborhoods in the $\tau=1$1-dimensional grid. For neighborhood radius $\tau=1$2 (typically a single cell), transition counts and state frequencies are aggregated over $\tau=1$3-neighborhoods: $\tau=1$4 State probability $\tau=1$5 is similarly softened. Empirically, this approach significantly enhances noise tolerance.

6. Parameter Selection and Practical Heuristics

Critical parameters for effective DDE-MGM implementation include:

Parameter	Heuristic/Default	Comment
Delay step $\tau=1$6	$\tau=1$7, $\tau=1$8 = dominant FFT index	Ensures sufficient coverage
Embedding $\tau=1$9	Use false-nearest-neighbor criterion	Increase until stability
Grid size $t$0	$t$150 bins/axis	Balances resolution/memory
Neighborhood $t$2	One grid cell	For robust matching

A plausible implication is that FFT-based estimation of $t$3 and nearest-neighbor checks for $t$4 streamline parameter selection without ad hoc tuning.

7. Computational Complexity and Memory Requirements

Each streaming sample requires:

• One finite-difference operation,
• One $t$5-vector shift,
• One grid assignment ($t$6),
• Two counter updates ($t$7 in practice via hashing/sparse tables).

Per-class memory is bounded by $t$8 state counts and a manageable set of active transitions. For $t$9 typical, $y'_t = (y_t - y_{t-\tau}) / \tau$02,500 floats plus a few thousand sparse transitions, enabling deployment in both embedded and high-throughput systems.

8. Illustrative Example in Embedding Space

Consider a 1-dimensional sinusoid with phase and offset drift. In raw space, different trials vary. After finite-difference, offset is removed and trials align in derivative space. Delay embedding into $y'_t = (y_t - y_{t-\tau}) / \tau$1 yields $y'_t = (y_t - y_{t-\tau}) / \tau$2 pairs tracing out planar loops. Overlaying a $y'_t = (y_t - y_{t-\tau}) / \tau$3 grid, each loop visits a subset of cells, producing observable statistics for both geographic (cell occupancy) and Markov (transition) models. During deployment, streaming input is continuously scored and classified with immediate feedback.

---

Classical delay embedding as formulated by Takens (“Detecting strange attractors in turbulence,” 1981) underpins the validity of DDE. The DDE-MGM scheme defined above achieves real-time, fully online classification of arbitrary-length time series and is invariant to offset, misalignment, and duration, with state-of-the-art empirical performance and computational efficiency (Zhang et al., 2016).