Randomized Time Warping (RTW)

• RTW is a set of sequence analysis methods defined by randomized subsequence sampling and subspace matching techniques that preserve DTW’s temporal elasticity.
• RTW replaces exhaustive dynamic programming with random sampling and PCA-driven canonical-angle comparisons, reducing computational cost and mitigating diagonal dominance.
• RTW comprises two main formulations: one for motion recognition using TE features and subspace analysis, and another using random warping series to build alignment-aware feature maps.

Randomized Time Warping (RTW) denotes a set of sequence-analysis methods that preserve the elastic-alignment motivation of Dynamic Time Warping (DTW) while replacing exhaustive path-based comparison with randomized constructions. In current arXiv usage, the term most directly refers to a general extension of DTW for sequence comparison and recognition that uses randomized, order-preserving sampling of subsequences, PCA-derived “hypo subspaces,” and canonical-angle similarity, with an interpretation as an attention mechanism in motion recognition (Hiraoka et al., 22 Aug 2025). A separate established usage realizes RTW through Random Warping Series (RWS), where each time series is aligned to short random warping series to obtain an explicit alignment-aware random-features embedding and a positive definite kernel (Wu et al., 2018). Across these formulations, randomness enters through sampled subsequences or sampled warping series rather than through a single deterministic optimal path.

1. Terminology and scope

The expression “Randomized Time Warping” is not attached to a single universally adopted formalism. At least two technical lineages are documented in the recent literature.

Formulation	Core mechanism	Representative source
Randomized Time Warping for motion recognition	Randomized, order-preserving sampling of subsequences; PCA hypo subspaces; canonical angles; attention interpretation	(Hiraoka et al., 22 Aug 2025)
Randomized Time Warping via Random Warping Series	Alignment to short random series; explicit feature map; positive definite kernel; Monte Carlo random features	(Wu et al., 2018)
RTW as acronym for a different concept	Riemannian manifold-aware multiple sequence alignment	(Richter et al., 2 Jun 2025)

This terminological plurality matters because the acronym RTW is not semantically unique. In “Riemannian Time Warping: Multiple Sequence Alignment in Curved Spaces,” RTW stands for Riemannian Time Warping, not randomized time warping (Richter et al., 2 Jun 2025). Two additional neighboring lines of work further widen the conceptual field: one studies estimation from randomly time-warped observations without naming a method RTW (Bayram, 2021), and another develops a stochastic process model for time warping functions that can be used to generate random warpings in a principled way (Ma et al., 2022).

A common source of confusion is therefore the assumption that RTW always denotes one algorithmic pipeline. The literature instead supports a narrower and more precise statement: RTW refers either to a randomized DTW extension built from sampled subsequences and subspace geometry (Hiraoka et al., 22 Aug 2025), or to an alignment-aware random-features construction built from random warping series (Wu et al., 2018), while other papers use the same acronym for unrelated manifold-based alignment (Richter et al., 2 Jun 2025).

2. DTW as the baseline that RTW generalizes or bypasses

Classical DTW aligns two sequences $X=(x_1,\dots,x_{T_X})$ and $Y=(y_1,\dots,y_{T_Y})$ by finding an admissible alignment path $P$ from $(1,1)$ to $(T_X,T_Y)$ that minimizes accumulated local dissimilarity: $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$ The admissible paths satisfy boundary conditions, monotonicity, and continuity, and the standard dynamic-programming solution operates on a $T_X\times T_Y$ cost matrix with typical time and space complexity $O(T_X\cdot T_Y)$, often written as $O(T^2)$ when the two lengths are comparable (Hiraoka et al., 22 Aug 2025).

The same baseline appears in the RWS literature in a slightly different notation. For time series $x=(x_1,\dots,x_n)$ and $Y=(y_1,\dots,y_{T_Y})$0, DTW is written as

$Y=(y_1,\dots,y_{T_Y})$1

with the dynamic-programming recurrence

$Y=(y_1,\dots,y_{T_Y})$2

plus standard boundary initialization and, optionally, a Sakoe–Chiba warping window $Y=(y_1,\dots,y_{T_Y})$3 (Wu et al., 2018).

The technical motivation for RTW follows from the computational and statistical limitations of DTW and its kernelized descendants. In the RWS account, DTW-inspired global-alignment kernels suffer from diagonal dominance of the Gram matrix and quadratic complexity with respect to the sample size, since evaluating all pairwise kernel values requires $Y=(y_1,\dots,y_{T_Y})$4 dynamic-programming runs, each with $Y=(y_1,\dots,y_{T_Y})$5 cost in the unwindowed case, for a total of $Y=(y_1,\dots,y_{T_Y})$6 time (Wu et al., 2018). In the motion-recognition account, DTW’s single optimal path is replaced by randomized sampling of many order-preserving subsequences, and the computational focus shifts from path search to subspace estimation and canonical-angle comparison (Hiraoka et al., 22 Aug 2025).

This suggests a unifying interpretation: RTW methods keep temporal elasticity as the central modeling objective, but they randomize the representation of possible warpings so that the algorithm need not commit to one explicit global path.

3. Randomized Time Warping as randomized subsequence sampling and subspace matching

In the motion-recognition formulation, RTW is a general extension of DTW that forgoes a single optimal warping path and instead represents many possible time-warped patterns through randomized, order-preserving sampling of subsequences called Time Elastic (TE) features (Hiraoka et al., 22 Aug 2025). The input is a sequence of $Y=(y_1,\dots,y_{T_Y})$7 deep feature vectors $Y=(y_1,\dots,y_{T_Y})$8, with $Y=(y_1,\dots,y_{T_Y})$9.

A TE feature is formed by sampling $P$0 indices $P$1 and concatenating the corresponding frame features: $P$2

$P$3

Repeating the randomized, order-preserving sampling $P$4 times yields a set $P$5, and an analogous set $P$6 is built for a reference sequence (Hiraoka et al., 22 Aug 2025). The stated effect is to induce a diverse collection of time-elastic patterns covering local to global scales, thereby approximating multiple warping possibilities that DTW would otherwise consider through path search.

The TE feature sets are then compressed into low-dimensional hypo subspaces by PCA. Writing

$P$7

the orthonormal basis matrices $P$8 and $P$9 are taken as the eigenvectors corresponding to the $(1,1)$0 largest eigenvalues of $(1,1)$1 and $(1,1)$2, respectively (Hiraoka et al., 22 Aug 2025). Structural similarity between the two subspaces is then measured with the Mutual Subspace Method (MSM): if

$(1,1)$3

and $(1,1)$4 are the singular values associated with the canonical angles $(1,1)$5, RTW defines

$(1,1)$6

The classification pipeline is explicitly two-stage. In the learning stage, each training video of class $(1,1)$7 is converted into TE features by repeated random order-preserving frame sampling and deep feature concatenation; PCA is applied to obtain a reference hypo subspace; and this process is repeated across the $(1,1)$8 training videos of the class to obtain $(1,1)$9. In the inference stage, the input video is processed by the same TE sampling procedure, PCA yields an input subspace $(T_X,T_Y)$0, MSM similarities are computed against the class-specific reference subspaces, and classification is performed by $(T_X,T_Y)$1-nearest neighbors using the average of the top-$(T_X,T_Y)$2 similarities within each class (Hiraoka et al., 22 Aug 2025).

The complexity profile differs sharply from DTW. TE generation requires $(T_X,T_Y)$3, PCA on $(T_X,T_Y)$4 and $(T_X,T_Y)$5 can dominate when $(T_X,T_Y)$6 and $(T_X,T_Y)$7 are large, and the MSM step itself is relatively light when the chosen subspace dimensions $(T_X,T_Y)$8 are small (Hiraoka et al., 22 Aug 2025). The paper explicitly contrasts this with DTW’s $(T_X,T_Y)$9 path-search cost and with Transformer self-attention’s $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$0 time and memory for the attention matrix.

4. Contribution weights, attention interpretation, and motion-recognition evidence

A distinctive claim of the 2025 formulation is that RTW’s core computational mechanism can be interpreted as a type of self-attention (Hiraoka et al., 22 Aug 2025). The derivation begins with explicit contribution weights connecting TE features, PCA, and canonical-angle computation back to per-time-step importance.

If $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$1 denotes the contribution of the $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$2-th TE feature to the $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$3-th canonical vector $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$4, the paper gives

$$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$5

where $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$6 and $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$7 are the linear-combination coefficients arising from PCA and MSM. If $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$8 denotes the contribution weight of the $$\mathrm{DTW}(X,Y)=\min_{P\in\Pi}\sum_{(i,j)\in P} d(x_i,y_j).$$9-th input element $T_X\times T_Y$0 to the $T_X\times T_Y$1-th TE feature $T_X\times T_Y$2, then

$T_X\times T_Y$3

where $T_X\times T_Y$4 and $T_X\times T_Y$5 is the total number of sampled TE features that include $T_X\times T_Y$6. The contribution of $T_X\times T_Y$7 to the $T_X\times T_Y$8-th canonical vector is then

$T_X\times T_Y$9

and the normalized RTW attention pattern is

$O(T_X\cdot T_Y)$0

with $O(T_X\cdot T_Y)$1 (Hiraoka et al., 22 Aug 2025).

This construction is compared to Transformer self-attention, where

$O(T_X\cdot T_Y)$2

and

$O(T_X\cdot T_Y)$3

The paper aggregates attention toward each element $O(T_X\cdot T_Y)$4 as $O(T_X\cdot T_Y)$5. The stated analogy is that each RTW pattern $O(T_X\cdot T_Y)$6 is a global weighting over the entire sequence, while the set $O(T_X\cdot T_Y)$7 parallels $O(T_X\cdot T_Y)$8 attention heads (Hiraoka et al., 22 Aug 2025). The key distinction is operational rather than superficial: RTW attention is computed over the entire input sequence through TE sampling and subspace analysis, whereas practical video Transformers often restrict attention to local or windowed views because naive self-attention requires an $O(T_X\cdot T_Y)$9 matrix.

On Something-Something V2, the empirical alignment between RTW and Transformer attention patterns is reported through cosine-similarity-based pairing of RTW canonical-angle patterns and Transformer heads. Averaged across 174 classes, the per-pair correlations are described as high, including values such as $O(T^2)$0, with overall average approximately $O(T^2)$1 across the ten smallest canonical angles (Hiraoka et al., 22 Aug 2025).

The same study reports a classification advantage for RTW over a ViViT-style Transformer baseline on SSv2. The dataset comprises 193k training and 27k test videos across 174 classes, with sequences averaging 40 frames, maximum 119, and standard deviation 10 (Hiraoka et al., 22 Aug 2025). The Transformer baseline uses a two-stage design with a ViT pre-trained on ImageNet-21k for spatial features and a temporal Transformer with 10 heads operating on 16-frame views. RTW uses the same spatial features, 10 canonical angles, and TE features built from 16-frame clips.

Method	Data fraction	Accuracy (%)
Transformer (ViViT-like)	Full	46.84
Transformer (ViViT-like)	1/5	47.53
Transformer (ViViT-like)	1/10	45.63
RTW $O(T^2)$2	Full	51.06
RTW $O(T^2)$3	1/5	48.29
RTW $O(T^2)$4	1/10	46.74
RTW $O(T^2)$5	Full	51.18
RTW $O(T^2)$6	1/5	48.90
RTW $O(T^2)$7	1/10	47.30
RTW $O(T^2)$8	Full	51.34
RTW $O(T^2)$9	1/5	49.00
RTW $x=(x_1,\dots,x_n)$0	1/10	47.65
RTW $x=(x_1,\dots,x_n)$1	Full	51.11
RTW $x=(x_1,\dots,x_n)$2	1/5	49.10
RTW $x=(x_1,\dots,x_n)$3	1/10	47.82

The reported full-data comparison yields about a 5% absolute improvement for RTW over the Transformer baseline, for example $x=(x_1,\dots,x_n)$4 versus $x=(x_1,\dots,x_n)$5 (Hiraoka et al., 22 Aug 2025). The same source states that RTW trained on only 10% of the dataset, at $x=(x_1,\dots,x_n)$6, approaches the Transformer’s full-data performance of $x=(x_1,\dots,x_n)$7. Another common misconception is therefore directly addressed by the paper’s formulation: RTW does not “learn” its attention weights through gradient-based optimization; the weights are induced analytically by randomized sampling, PCA, and SVD (Hiraoka et al., 22 Aug 2025).

5. Random Warping Series: RTW as an alignment-aware random-features method

A distinct and earlier formulation of Randomized Time Warping is given by the Random Warping Series framework (Wu et al., 2018). Here the central object is not a hypo-subspace representation but an explicit feature map built by aligning each time series to a collection of short random series $x=(x_1,\dots,x_n)$8.

The kernel is defined as

$x=(x_1,\dots,x_n)$9

with feature map

$Y=(y_1,\dots,y_{T_Y})$00

A crucial choice is the “hard alignment” distribution that places all mass on the optimal alignment: $Y=(y_1,\dots,y_{T_Y})$01 The same feature can be written using alignment operators $Y=(y_1,\dots,y_{T_Y})$02 as

$Y=(y_1,\dots,y_{T_Y})$03

where $Y=(y_1,\dots,y_{T_Y})$04 is a ground dissimilarity such as $Y=(y_1,\dots,y_{T_Y})$05 or squared $Y=(y_1,\dots,y_{T_Y})$06 applied per aligned pair (Wu et al., 2018).

Monte Carlo approximation then produces an explicit $Y=(y_1,\dots,y_{T_Y})$07-dimensional embedding: $Y=(y_1,\dots,y_{T_Y})$08 with $Y=(y_1,\dots,y_{T_Y})$09 (Wu et al., 2018). The sampled random series follow a specified distribution: the length $Y=(y_1,\dots,y_{T_Y})$10 is drawn uniformly on $Y=(y_1,\dots,y_{T_Y})$11, commonly with $Y=(y_1,\dots,y_{T_Y})$12 and $Y=(y_1,\dots,y_{T_Y})$13; the values are i.i.d. Gaussian with zero mean and variance $Y=(y_1,\dots,y_{T_Y})$14; and a Sakoe–Chiba band such as $Y=(y_1,\dots,y_{T_Y})$15 can be used to constrain the DTW alignment (Wu et al., 2018).

The theoretical claims are twofold. First, positive definiteness follows directly from the outer-product construction: $Y=(y_1,\dots,y_{T_Y})$16 Second, the paper proves a uniform-convergence result for the random-features approximation on domains of time series of unbounded length, expressed in terms of a Minimum Shape-Preserving Length $Y=(y_1,\dots,y_{T_Y})$17 and a sufficient number of features

$Y=(y_1,\dots,y_{T_Y})$18

with the bounded-length case giving $Y=(y_1,\dots,y_{T_Y})$19 (Wu et al., 2018).

The computational advantage is explicit. DTW between a series of length $Y=(y_1,\dots,y_{T_Y})$20 and a short random series of length $Y=(y_1,\dots,y_{T_Y})$21 costs $Y=(y_1,\dots,y_{T_Y})$22, which behaves as $Y=(y_1,\dots,y_{T_Y})$23 when $Y=(y_1,\dots,y_{T_Y})$24 is small or sampled from a small range. For $Y=(y_1,\dots,y_{T_Y})$25 random series and $Y=(y_1,\dots,y_{T_Y})$26 samples, the total cost is $Y=(y_1,\dots,y_{T_Y})$27 time and $Y=(y_1,\dots,y_{T_Y})$28 memory, in contrast to the $Y=(y_1,\dots,y_{T_Y})$29 time and $Y=(y_1,\dots,y_{T_Y})$30 memory of DTW-based global-alignment kernels (Wu et al., 2018). The paper also argues that the hard-min construction avoids diagonal dominance because the feature $Y=(y_1,\dots,y_{T_Y})$31 depends on the single optimal alignment rather than a soft sum over exponentially many paths.

Empirically, the RWS study reports experiments on 16 UCR benchmark datasets for classification and clustering. Concrete examples include Two_Patterns, where RWS(SR) achieves $Y=(y_1,\dots,y_{T_Y})$32–$Y=(y_1,\dots,y_{T_Y})$33 accuracy in $Y=(y_1,\dots,y_{T_Y})$34–$Y=(y_1,\dots,y_{T_Y})$35 seconds versus $Y=(y_1,\dots,y_{T_Y})$36–$Y=(y_1,\dots,y_{T_Y})$37 seconds for 1NN-DTW, and Wafer, where RWS(SR) achieves $Y=(y_1,\dots,y_{T_Y})$38 in $Y=(y_1,\dots,y_{T_Y})$39 seconds versus $Y=(y_1,\dots,y_{T_Y})$40 in $Y=(y_1,\dots,y_{T_Y})$41 seconds for 1NN-DTW (Wu et al., 2018). Against TGAK, the reported contrasts are stronger: on Wafer, RWS(SR) obtains $Y=(y_1,\dots,y_{T_Y})$42 in $Y=(y_1,\dots,y_{T_Y})$43 seconds versus $Y=(y_1,\dots,y_{T_Y})$44 in $Y=(y_1,\dots,y_{T_Y})$45 seconds for TGAK; on MALLAT, the comparison is $Y=(y_1,\dots,y_{T_Y})$46 in $Y=(y_1,\dots,y_{T_Y})$47 seconds versus $Y=(y_1,\dots,y_{T_Y})$48 in $Y=(y_1,\dots,y_{T_Y})$49 seconds (Wu et al., 2018). The paper’s practical guidance recommends $Y=(y_1,\dots,y_{T_Y})$50, $Y=(y_1,\dots,y_{T_Y})$51, $Y=(y_1,\dots,y_{T_Y})$52, $Y=(y_1,\dots,y_{T_Y})$53, and a modest Sakoe–Chiba window.

6. Related stochastic models, adjacent formulations, and naming ambiguities

Several nearby papers study random time warpings without using the exact RTW formalism of either subsequence/subspace RTW or RWS. These works are useful for delimiting what Randomized Time Warping is and is not.

“Time-Series Estimation from Randomly Time-Warped Observations” considers the observation model

$Y=(y_1,\dots,y_{T_Y})$54

with random warpings $Y=(y_1,\dots,y_{T_Y})$55 that are close to identity, for example $Y=(y_1,\dots,y_{T_Y})$56 where $Y=(y_1,\dots,y_{T_Y})$57 is drawn from a zero-mean stationary Gaussian process (Bayram, 2021). The paper’s proposed estimator does not use DTW; instead it constructs a sparse graph Laplacian over observed signals, repeatedly solves discrete Dirichlet problems with random boundary seeds, defines centrality by inverse temperature variance, and estimates the template by averaging the most central observations. It further shows that naive ensemble averaging converges to a blurred version $Y=(y_1,\dots,y_{T_Y})$58, which motivates selecting near-unwarped observations rather than averaging all warped ones (Bayram, 2021). The paper explicitly states that it does not use “RTW” as a named method; RTW there refers only to the generative setting of random time warpings.

“A Stochastic Process Model for Time Warping Functions” also does not explicitly discuss Randomized Time Warping by name, but it supplies a mathematically rigorous way to generate random warpings (Ma et al., 2022). The key construction maps a warping function $Y=(y_1,\dots,y_{T_Y})$59 to the centered logratio transform of its derivative,

$Y=(y_1,\dots,y_{T_Y})$60

which places the problem in a zero-mean $Y=(y_1,\dots,y_{T_Y})$61 Hilbert subspace. Sampling a finite stochastic expansion

$Y=(y_1,\dots,y_{T_Y})$62

then induces a valid random warping through

$Y=(y_1,\dots,y_{T_Y})$63

The paper presents this as a generative model for warping functions in $Y=(y_1,\dots,y_{T_Y})$64, supports functional PCA, ANOVA, and regression in the transformed space, and develops a Bayesian registration method with efficient gradient-based MAP estimation (Ma et al., 2022). A plausible implication is that this framework can serve as a principled prior or generator for RTW-style randomization, although the paper itself does not define such an algorithm.

Finally, the overloaded acronym RTW must be distinguished from “Riemannian Time Warping: Multiple Sequence Alignment in Curved Spaces,” where RTW denotes a manifold-aware method for multiple sequence alignment on Riemannian manifolds (Richter et al., 2 Jun 2025). That method performs windowed sinc interpolation in tangent spaces, computes a Karcher mean trajectory by Gauss–Newton, and optimizes monotone warpings with a Gaussian-windowed geodesic loss and monotonicity penalty. It is therefore related to time warping at the problem level, but not to randomized time warping in the sense of sampled subsequences or random warping series.

Taken together, these papers delimit the encyclopedia meaning of Randomized Time Warping. In the strictest sense, RTW refers to randomized constructions that replace or approximate explicit alignment search, either through TE-feature sampling and canonical-angle subspace comparison in video recognition (Hiraoka et al., 22 Aug 2025) or through random-warping-series feature maps and positive definite kernels in time-series embedding (Wu et al., 2018). In a broader sense, the adjacent literature on random warp generative models and randomly time-warped observations provides the probabilistic and inferential background against which those RTW methods can be understood (Bayram, 2021, Ma et al., 2022).