Hybrid Temporal & Multivariate Embeddings

• HTME is an embedding paradigm that separates temporal and multivariate feature extraction before fusing them, enabling tailored model designs for complex time-series data.
• The method improves prediction accuracy by isolating feature streams, as shown by significant gains in MSE and MAE across varied datasets.
• HTME architectures support diverse approaches including Transformer, ConvLSTM, and graph attention for handling both spatial and temporal dependencies.

Searching arXiv for the named HTME paper and closely related hybrid temporal/multivariate embedding work. Hybrid Temporal and Multivariate Embeddings (HTME) denotes an embedding paradigm for multivariate time-series and spatiotemporal learning in which temporal structure and cross-variable, cross-channel, or cross-region structure are modeled through distinct but coordinated representational pathways, then fused into a single latent representation. In the literature, the term is used explicitly by HTMformer, which defines HTME as an embedding extractor that combines a temporal feature extraction module and a multivariate feature extraction module before Transformer encoding (Wang et al., 8 Oct 2025). Closely related work applies the same underlying principle without always using the exact term: InvDec separates temporal encoding from variate-level decoding for forecasting (Wang, 23 Oct 2025), B-TGAT combines ConvLSTM2D-based local spatiotemporal encoding with a graph-attention temporal bottleneck for clustering climate sequences (Nji et al., 16 Sep 2025), and HiTime constructs hierarchical temporal embeddings that are then aligned with semantic text embeddings for multivariate time-series classification (Tao et al., 2024). Taken together, these works establish HTME as a design pattern for learning latent spaces that are temporally informative, multivariate-aware, and often task-adaptive.

1. Conceptual definition and scope

HTME emerged from a recurring limitation in sequence models for multivariate data: architectures that are strong at temporal modeling often underuse cross-variate structure, while architectures that emphasize inter-variable relations may weaken temporal encoding. InvDec states this trade-off directly: channel-independent models such as PatchTST are strong at temporal modeling but ignore variable correlations, whereas inverted, variate-attention models such as iTransformer model variable relationships well but sacrifice temporal attention (Wang, 23 Oct 2025). HTMformer makes an analogous argument at the embedding level, asserting that existing Transformers for time series “tend to overemphasize temporal dependencies,” while inter-variable correlations in traffic, weather, electricity, and related settings are also highly informative (Wang et al., 8 Oct 2025).

Within this framing, HTME is not a single architecture but a family of representational strategies with a common principle: one component learns temporal dynamics, another learns multivariate structure, and a fusion mechanism combines them. In HTMformer, the temporal and multivariate branches are explicit modules fused by a learnable scalar $\alpha$ (Wang et al., 8 Oct 2025). In InvDec, the separation is staged rather than parallel: temporal embeddings are formed first, then delayed variate embeddings and variate-wise self-attention create a distinct multivariate representation, and the two streams are combined through adaptive residual fusion (Wang, 23 Oct 2025). In B-TGAT, ConvLSTM2D and U-Net-style compression learn localized spatiotemporal structure, while a graph-attention bottleneck transforms the deepest latent tensor into regime embeddings that capture broader spatial and temporal dependencies (Nji et al., 16 Sep 2025).

A plausible implication is that HTME is best understood as a representational doctrine rather than a single formal model class. The common requirement is that temporal information and multivariate information are not collapsed prematurely into one undifferentiated token stream.

2. Core architectural pattern

The most explicit formulation appears in HTMformer, where HTME is defined as an embedding extractor with two independent modules: a Temporal Feature Extractor and a Multivariate Feature Extractor (Wang et al., 8 Oct 2025). After RevIN normalization and timestamp concatenation, the pipeline is

$X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$

with $X \in \mathbb{R}^{L \times C}$, $Y_{\text{in} \in \mathbb{R}^{N \times D}$, and fusion defined as

$Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$

where $D_{\text{out}$ is the temporal branch output and $V_{\text{out}$ is the multivariate branch output (Wang et al., 8 Oct 2025). This is the clearest direct specification of HTME in the current literature.

InvDec implements the same principle sequentially rather than symmetrically. For each variable $c$, the series $\mathbf{x}_c \in \mathbb{R}^L$ is segmented into patches, projected into $D$-dimensional tokens, and encoded with self-attention over reshaped patch tokens, yielding a temporal representation $X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$0 (Wang, 23 Oct 2025). Only after temporal encoding does the model aggregate to one token per variable by global pooling,

$X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$1

and apply variate-wise self-attention to model cross-variable dependencies (Wang, 23 Oct 2025). This “delayed variate embedding” design is central to its interpretation as HTME because variable identity is injected after temporal feature extraction rather than at the input.

B-TGAT generalizes the pattern to spatiotemporal tensors. Its encoder consumes a climate tensor $X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$2 through stacked TimeDistributed ConvLSTM2D blocks and pooling, then reshapes the deepest tensor into graph node features for each time slice, $X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$3 with $X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$4 (Nji et al., 16 Sep 2025). The bottleneck is therefore both temporally indexed and spatially graph-structured. This suggests a broader interpretation of HTME in which “multivariate” includes not only channel relations but also graph-based region interactions.

3. Temporal embedding mechanisms

HTME methods differ substantially in how they construct temporal embeddings, but they share the assumption that temporal structure should be learned with dedicated operators rather than delegated entirely to a generic downstream attention stack.

HTMformer uses patching, convolution, flattening, and linear projection as its temporal feature extraction module. The paper states that the series is split into patches; each patch is treated as a token; a convolution extracts short-term temporal patterns; the output is flattened and linearly projected; and another projection along the temporal dimension models long-range correlations (Wang et al., 8 Oct 2025). The branch is formalized as

$X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$5

$X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$6

$X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$7

(Wang et al., 8 Oct 2025).

InvDec instead inherits a patch-based temporal encoder from PatchTST. For each variable, non-overlapping patches are projected as

$X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$8

and standard self-attention and FFN blocks operate over the reshaped patch tokens (Wang, 23 Oct 2025). The paper emphasizes that this preserves local temporal context while reducing sequence length and computational cost.

HiTime uses a hierarchical feature encoder with two Transformer-based branches: a data-specific encoder trained self-supervised through tokenization, masking, and reconstruction, and a task-specific encoder trained with labels to capture task-relevant long-term and short-term dependencies (Tao et al., 2024). Their concatenation,

$X_{\text{in} = \text{CAT}(X,T), \qquad Y_{\text{in} = \text{HTMEE}(X_{\text{in}), \qquad Y_{\text{out} = \text{Encode}(Y_{\text{in}), \qquad Y = \text{Project}(Y_{\text{out})$9

is explicitly presented as the main feature-level hybrid embedding (Tao et al., 2024). Here the temporal split is not between short-term and long-term modules in a single encoder, but between general temporal structure and task-discriminative temporal structure.

A more specialized antecedent is MEANTIME, which uses six temporal embeddings—three absolute and three relative—and assigns different temporal embeddings to different attention heads (Cho et al., 2020). Its absolute embeddings are Day, Pos, and Con; its relative embeddings are Sin, Exp, and Log (Cho et al., 2020). Although the paper is in sequential recommendation rather than general forecasting, it is a clear precursor to HTME-style multi-view temporal encoding because it rejects the sufficiency of a single positional embedding.

4. Multivariate embedding mechanisms

The multivariate component of HTME is the modeling of correlations across variables, channels, sensors, spatial nodes, or regions. In HTMformer, the multivariate feature extraction module is deliberately lightweight. The input is patched, flattened, passed through a linear layer to model inter-variable correlations, aggregated through GRU, and expanded by convolution:

$X \in \mathbb{R}^{L \times C}$0

$X \in \mathbb{R}^{L \times C}$1

$X \in \mathbb{R}^{L \times C}$2

(Wang et al., 8 Oct 2025). The paper explicitly characterizes this as a “weak learning” strategy for multivariate correlation modeling.

InvDec adopts a more direct variate-attention approach. After pooling patch-level temporal encodings into one token per variable, the decoder computes learnable variate-wise self-attention:

$X \in \mathbb{R}^{L \times C}$3

$X \in \mathbb{R}^{L \times C}$4

(Wang, 23 Oct 2025). The decoder output is then projected and broadcast back across the patch dimension. This makes the variate dimension the primary axis of attention in the decoder.

B-TGAT uses graph structure rather than channel attention to encode multivariate or spatial interactions. At the bottleneck, a graph $X \in \mathbb{R}^{L \times C}$5 is built over spatial nodes using spatial adjacency or $X \in \mathbb{R}^{L \times C}$6-NN structure, and graph attention is applied as

$X \in \mathbb{R}^{L \times C}$7

(Nji et al., 16 Sep 2025). This allows adaptive weighting of neighbors rather than fixed spatial connectivity. The paper’s interpretation is that the bottleneck should not be treated as a mere vector; it should be interpreted as a spatiotemporal graph.

An even broader extension appears in HSMGNN, which embeds multivariate time series jointly in Euclidean space and on the SPD manifold through SCS embedding, then learns graph structure in both spaces before weighted fusion (Fang et al., 16 Dec 2025). This suggests that HTME can extend beyond temporal-versus-variate separation to hybrid geometric representations of multivariate dependencies.

5. Fusion strategies and learning objectives

A defining property of HTME methods is that temporal and multivariate information are not merely extracted independently; they are recombined through explicit fusion rules, and training objectives are chosen so that the fused latent space remains useful for the target task.

HTMformer uses the simplest fusion rule: a learnable scalar $X \in \mathbb{R}^{L \times C}$8 combines the two branches,

$X \in \mathbb{R}^{L \times C}$9

with the stated interpretation that datasets with stronger temporal structure can rely more on $Y_{\text{in} \in \mathbb{R}^{N \times D}$0, while datasets with richer inter-variable coupling can benefit more from $Y_{\text{in} \in \mathbb{R}^{N \times D}$1 (Wang et al., 8 Oct 2025).

InvDec uses adaptive residual fusion,

$Y_{\text{in} \in \mathbb{R}^{N \times D}$2

where $Y_{\text{in} \in \mathbb{R}^{N \times D}$3 is a learnable or preset weight (Wang, 23 Oct 2025). The paper explicitly states that smaller $Y_{\text{in} \in \mathbb{R}^{N \times D}$4 is preferable for low-dimensional datasets, while larger $Y_{\text{in} \in \mathbb{R}^{N \times D}$5 is more useful for high-dimensional datasets. This makes dimensionality a central condition governing temporal-versus-variate balance.

B-TGAT fuses representational aims through a joint objective that combines reconstruction and deep clustering. The decoder reconstructs the input sequence under

$Y_{\text{in} \in \mathbb{R}^{N \times D}$6

while the clustering head applies Student’s $Y_{\text{in} \in \mathbb{R}^{N \times D}$7-distribution soft assignments $Y_{\text{in} \in \mathbb{R}^{N \times D}$8, target distribution $Y_{\text{in} \in \mathbb{R}^{N \times D}$9, and KL divergence

$Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$0

The total objective is

$Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$1

(Nji et al., 16 Sep 2025). In this formulation, HTME is not only about representation fusion but also about latent-space shaping: the embedding must remain reconstructive and cluster-friendly.

HiTime adds a multimodal alignment dimension. After hierarchical temporal embedding generation, it aligns time-series and text embeddings via fine-grained and coarse-grained objectives, with total loss

$Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$2

(Tao et al., 2024). This suggests that HTME can be extended into a tri-partite scheme: temporal embedding, multivariate-aware latent structure, and semantic alignment.

6. Empirical results and application domains

The empirical literature around HTME spans forecasting, clustering, classification, recommendation, and multimodal geospatial vision. The reported results consistently support the claim that separate temporal and multivariate embedding pathways are beneficial, especially as dimensionality or structural complexity increases.

Method	Domain	Reported result
HTMformer	Time series forecasting	First in 26/40 MSE metrics and 29/40 MAE metrics on eight datasets (Wang et al., 8 Oct 2025)
InvDec-PatchTST	Multivariate forecasting	20.9% MSE reduction on Electricity, 4.3% improvement on Weather, 2.7% gain on Traffic over PatchTST (Wang, 23 Oct 2025)
B-TGAT	Climate clustering	Best reported Silhouette, DB, I-CD, and RMSE on ERA5; best reported Silhouette, DB, CH, RMSE, and Variance on CARRA; best reported Silhouette, CH, RMSE, and I-CD on NCEP/NCAR (Nji et al., 16 Sep 2025)
HiTime	MTSC with LLMs	Average accuracy 0.8764 versus 0.8452 for the next best reported competitor, Reformer (Tao et al., 2024)
MEANTIME	Sequential recommendation	Best on all datasets and metrics, with up to 16.03% relative gain in NDCG@5 on Game (Cho et al., 2020)

HTMformer is evaluated on Electricity, Weather, Traffic, ETTh2, Solar-Energy, PEMS03, PEMS04, and PEMS08, and is reported to be especially strong on high-dimensional datasets such as Traffic and Solar-Energy (Wang et al., 8 Oct 2025). It also functions as a plug-in embedding strategy for Transformer, Reformer, Informer, Flowformer, and Flashformer, with average reported improvements of 35.8%, 34.6%, 43.6%, 31.9%, and 33.1%, respectively (Wang et al., 8 Oct 2025).

InvDec’s dimensionality analysis is particularly significant for the HTME concept. The paper states that the relative improvement over PatchTST increases with variable count, that cross-variate modeling becomes more valuable as the number of variables increases, and that gains are small on low-dimensional ETT datasets with $Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$3 (Wang, 23 Oct 2025). This is one of the clearest empirical statements that the need for HTME is data-regime dependent rather than universal.

B-TGAT demonstrates that the same design principle applies beyond forecasting. On ERA5 it reports Silhouette 0.3268, DB 1.5009, I-CD 7.4839, and RMSE 13.2158 as best reported values; on CARRA it reports Silhouette 0.2767, DB 1.5089, CH 69.7729, RMSE 5.5424, and Variance 0.0105; on NCEP/NCAR it reports Silhouette 0.6541, CH 868.7555, RMSE 3.118, and I-CD 0.9098 (Nji et al., 16 Sep 2025). The paper interprets these results as evidence of superior cluster separability, temporal stability, and alignment with known climate transitions.

7. Interpretation, related ideas, and open questions

The current literature supports several interpretations of HTME, while also revealing unresolved design questions.

First, HTME is not synonymous with “adding variable embeddings.” InvDec’s delayed variate embeddings are explicitly motivated by preserving temporal feature integrity, in contrast to injecting variable embeddings at the input stage as in TimeXer (Wang, 23 Oct 2025). This suggests that the order in which temporal and multivariate signals are introduced matters. A common misconception is that any model with time tokens and channel tokens is already hybrid; the cited work instead emphasizes principled separation, then controlled fusion.

Second, HTME is not restricted to Transformer architectures. B-TGAT uses ConvLSTM2D, U-Net skip connections, graph attention, BiLSTM, and DEC-style clustering (Nji et al., 16 Sep 2025). The 2023 “Temporal Embeddings” work for geospatial computer vision transforms activity time series to the frequency domain, compresses them with a contractive autoencoder, and outputs image-like channels for multimodal fusion with RGB, SAR, and graph embeddings (Cao et al., 2023). This suggests that HTME can also be instantiated through spectral encoders, autoencoders, or graph pipelines rather than only through token self-attention.

Third, there is an unresolved question about the optimal fusion mechanism. Current methods use scalar weighting, residual addition, concatenation, or objective-level coupling. HTMformer uses a learnable scalar $Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$4 (Wang et al., 8 Oct 2025); InvDec uses residual fusion with $Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$5 (Wang, 23 Oct 2025); B-TGAT relies on the interaction of reconstruction and clustering losses (Nji et al., 16 Sep 2025); HSMGNN uses weighted dual-space fusion with $Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$6 and $Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$7 (Fang et al., 16 Dec 2025). This suggests that fusion remains method-specific rather than standardized.

Fourth, empirical evidence repeatedly indicates that hybridization is most beneficial in structurally rich settings. InvDec’s advantage grows with dataset dimensionality (Wang, 23 Oct 2025). HTMformer is especially strong on high-dimensional datasets and can reduce the complexity of attention from $Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$8 to $Y_{\text{in} = \alpha D_{\text{out} + (1-\alpha) V_{\text{out}$9 by using inverted input after HTME extraction (Wang et al., 8 Oct 2025). B-TGAT is designed for high-dimensional multivariate spatiotemporal climate tensors where both local and global temporal dependencies matter (Nji et al., 16 Sep 2025). A plausible implication is that HTME is most valuable when neither pure temporal encoding nor pure variate encoding can dominate.

Finally, the literature indicates several converging trajectories for future HTME research. One direction is multimodal semantic alignment, exemplified by HiTime’s dual-view alignment between time-series and text embeddings (Tao et al., 2024). Another is hybrid geometry, exemplified by HSMGNN’s Euclidean-SPD manifold formulation (Fang et al., 16 Dec 2025). A third is stronger task-dependent latent shaping, as in B-TGAT’s reconstruction-plus-clustering objective (Nji et al., 16 Sep 2025). These lines of work suggest that HTME may continue to evolve from a two-stream embedding idea into a broader framework for structured latent-space design across time, variables, space, modality, and geometry.