QuITE: Query-Based Irregular Time Series Embedding

Abstract: Irregular Multivariate Time Series (IMTS) are common in practice, yet their irregular sampling complicates effective modeling. Existing approaches typically either (i) design specialized architectures that limit the reuse of proven Multivariate Time Series (MTS) models, or (ii) map IMTS onto regular temporal grids through interpolation, which may distort temporal dynamics by introducing artificial values. To address these limitations, we propose a new input-embedding-based approach. We identify that the key bottleneck lies not in the backbone architecture, but in conventional embedding layers that assume uniform sampling. In this work, we introduce QuITE (Query-Based Irregular Time Series Embedding), a simple yet effective plug-and-play embedding module for IMTS. QuITE employs learnable query tokens to aggregate irregular observations through a single self-attention layer, directly producing backbone-compatible latent representations without artificial value generation or architectural modification. Extensive experiments on real-world benchmarks show that QuITE consistently improves MTS models, yielding average relative gains of up to $54.7\%$ in forecasting and $15.8\%$ in classification across diverse datasets and backbone architectures. Code is available at: https://github.com/Meaningfull9502/QuITE.

• The paper proposes a novel query-based embedding scheme that aggregates irregular observations using learnable self-attention queries, eliminating interpolation artifacts.
• The method combines harmonic time encodings with linear value embeddings to create structured latent tokens compatible with any multivariate time series backbone.
• Empirical results demonstrate notable gains in forecasting and classification, achieving up to 54.7% improvement in MSE and robust performance under >75% missingness.

Query-Based Embedding for Irregular Multivariate Time Series: An Expert Analysis of QuITE

Introduction and Motivation

Irregular Multivariate Time Series (IMTS) are pervasive in domains like healthcare (EHR, ICU data), climate science, and human activity monitoring. The key challenge with IMTS is the lack of temporal alignment: variables are sampled asynchronously with non-uniform intervals, leading to missing values and substantial temporal irregularity. Existing IMTS modeling approaches either design bespoke architectures tailored to irregularity, sacrificing reuse of powerful multivariate time series (MTS) backbones, or resort to value interpolation, introducing artifacts that obscure true signal structure. The "QuITE: Query-Based Irregular Time Series Embedding" (2605.28166) paper attacks this problem from a previously underexplored axis: instead of altering the backbone or the data, it rethinks the embedding layer, hypothesizing that legacy input embeddings are the principal bottleneck preventing robust IMTS modeling by standard MTS networks.

QuITE: Embedding IMTS via Learnable Query Aggregation

QuITE proposes an input-embedding module that is both plug-and-play and backbone-agnostic. The core technical innovation is to introduce a set of learnable query tokens, each designed to serve as a structured aggregation anchor over irregular observations. Concretely, the QuITE embedding converts irregular value–timestamp–mask triplets into latent representations using:

• Time Embedding: Harmonic time encodings parameterized by learnable frequencies and phases (cf. mTAND architecture), effectively mapping continuous timestamps to latent space.
• Value Embedding: Linear encoding of observed values.

Each observation becomes a token; the tokens for each variable (or patch, in local modeling) are aggregated by a small set of learnable queries via a single self-attention layer, where masking naturally handles missingness. This produces variable- or patch-level embeddings compatible with any downstream MTS backbone. By design, QuITE eliminates the need for data interpolation and preserves temporal information without lossy pooling.

Figure 1: QuITE framework: learnable queries aggregate irregular observations via self-attention, producing structured embeddings for each variable (or patch).

QuITE$^{++}$: Hierarchical Query-Based Encoder

Building on the embedding-level methodology, QuITE$^{++}$ introduces a hierarchical encoder/decoder structure, suitable for sequence forecasting. The encoder first aggregates patches via variable-specific tokens, then models inter-variable dependencies using variable-level self-attention. For forecasting, the decoder employs future timestamp queries—embedded as in QuITE—to attend over both local (patch-level) and global (variable-level) representations, yielding flexible, query-aware predictions for arbitrary forecast horizons.

Figure 2: QuITE$^{++}$ architecture: stacked patch-level and variable-level attention mediates hierarchical aggregation; decoder cross-attends to past representations using future time queries.

Numerical Results

Empirical evaluation spans multiple real-world benchmarks in both forecasting and classification, with simulation of severe sampling irregularity and missingness ratios often exceeding 75%. The main findings are:

• Forecasting: Average relative gains up to 54.7% in MSE for variate-token models (iTransformer, S-Mamba) and 5–16% for patch-token and hybrid models, after integrating QuITE into SOTA MTS backbones.
• Classification: Consistent improvements (5.3%–15.8% average relative gain) on AUROC, AUPRC, and accuracy metrics across all backbone/model classes.

These improvements are robust across diverse tasks (biomechanical sensors, clinical data, climate records) and persist under high rates of artificial observation removal, until surpassing ~75% sparsity.

Figure 3: Forecasting: QuITE consistently improves performance across all backbone families and datasets.

Embedding Quality and Ablation

The benefit of QuITE at the representation level is confirmed by t-SNE visualizations: QuITE produces embedding spaces with tighter, more discriminative class clusters relative to conventional or mean-pooled embeddings. Ablation over input embedding variants (Add, Concat, mTAND, mean pooling, QuITE) demonstrates that QuITE achieves the best or second-best forecasting MSE/MAE in every case, outperforming all strong alternatives including latent-grid approaches such as mTAND.

Figure 4: Hyperparameter sensitivity: model performance is robust across reasonable ranges of hidden dimension, layers, and attention heads.

Comparison Against IMTS-Specific Models

Benchmarking includes 17 SOTA baselines, ranging from continuous-time ODE-RNN approaches (Latent-ODE) and graph-hypergraph methods (tPatchGNN, HyperIMTS, GraFITi) to specialized patch/attention models (Hi-Patch, Warpformer). QuITE-equipped MTS backbones compete favorably against even the strongest of these, with QuITE$^{++}$ itself delivering the best overall performance in 20 of 24 settings. Notably, this is accomplished with lower parameter count and higher modularity, as QuITE can be paired with any standard backbone.

Theoretical and Practical Implications

QuITE and QuITE$^{++}$ decouple IMTS modeling performance from the necessity of architectural overhaul or interpolation. This plug-and-play approach enables direct reuse of validated, highly optimized MTS models, which is especially practical in industrial and clinical deployments with stringent system constraints and extensive legacy codebases. The model's ability to act as a latent set function—aggregating unordered, masked, irregular data into structured tokens while retaining full compatibility with patch-based, variate-based, or hybrid architectures—opens the door for wider adoption of modern sequence models in sensitive, highly irregular domains such as EHR or remote sensing. Furthermore, the clear empirical superiority under severe irregularity and missingness suggests that future research on temporal deep learning for real-world applications should pay greater focus to embedding-level inductive biases.

Future Directions

Extensions of QuITE could include: (i) integration with diffusion models and continuous-time state-space backbones; (ii) hybridization with metric learning or self-supervised pre-training for irregular unlabeled data; (iii) adaptive query initialization using prior domain knowledge (e.g., variable functional roles in biomedicine); and (iv) exploration of context-dependent query tokenization (e.g., attention over hierarchical or multimodal variable sets). Methodologically, QuITE suggests a new generation of set-based embedding techniques, potentially informed by permutation-invariant deep sets theory or generative models for set-valued observations.

Conclusion

QuITE offers a general, effective, and computationally efficient mechanism for embedding irregular multivariate time series, enabling the full power of SOTA multivariate time series backbones for irregular settings without architectural changes or interpolation. Extensive experiments validate substantial and robust empirical gains, while architecture analysis and scalability profiling show favorable complexity–accuracy tradeoffs. The QuITE paradigm substantially advances practical modeling of IMTS and is poised for adoption in high-impact domains.