TiRex-2: Generalizing TiRex to Multivariate Data and Streaming

Abstract: We introduce TiRex-2, a recurrent xLSTM-based time series foundation model that generalizes the univariate TiRex to multivariate forecasting with both past and future covariates. Real-world forecasting is inherently sequential: observations arrive continuously, variables evolve jointly, and a subset of covariates is known ahead of time. Existing Transformer-based time series foundation models capture cross-variate dependencies but incur quadratic complexity in context length and require full-history recomputation as new observations arrive. TiRex-2 addresses these limitations through a memory-centric recurrent design that operates at constant per-patch cost under streaming. The model combines a bidirectional time mixer with an asymmetric grouped-attention variate mixer, enabling the integration of future-known covariates while preserving strict causality over target variables. To our knowledge, this is the first time series foundation model that achieves this combination of properties. To support scalable multivariate pretraining, we propose a synthetic coupling pipeline that composes diverse multivariate samples on the fly from large univariate corpora. Empirically, TiRex-2 achieves state-of-the-art zero-shot performance on GIFT-Eval and fev-bench, remains stable when streamed to arbitrary context lengths, and maintains constant inference cost per patch. The model uses 38.4M active parameters in univariate mode, with an additional 44.1M parameters activated for multivariate forecasting.

• The paper demonstrates that a memory-centric, xLSTM architecture effectively integrates multivariate covariates and enforces strict target causality in streaming settings.
• It leverages a synthetic coupling pipeline and asymmetric attention masking to generate universal cross-variate representations at reduced computational cost.
• TiRex-2 achieves superior zero-shot accuracy and maintains constant per-update cost, proving its viability for real-time, resource-constrained applications.

TiRex-2: Generalizing Recurrent Foundation Models to Multivariate and Streaming Time Series Forecasting

Introduction

TiRex-2 addresses outstanding limitations of time series foundation models (TSFMs) for real-world forecasting applications. The shift from univariate to genuinely multivariate, covariate-aware contexts is critical for domains where system state is defined by multiple interacting variates (e.g., industrial monitoring, earth sciences, cloud operations). Prevailing approaches achieve cross-variate modeling with Transformer-based architectures, but at prohibitive costs for inference in streaming settings and with full-context recomputation at each new observation. TiRex-2 proposes a memory-centric, xLSTM-based paradigm that unifies long-context, zero-shot generalization with efficient, strictly causal streaming in arbitrary multivariate configurations, while incorporating both past and future-known covariates.

Figure 1: Comparative overview highlighting that TiRex-2 combines native multivariate covariate handling, strict target-causality, and constant-memory streaming—capabilities absent in Chronos-2 and the original TiRex.

Architectural Innovations

TiRex-2 is built on a modular, alternating stack of bidirectional time mixers (xLSTM blocks) and asymmetric grouped-attention variate mixers. After instance normalization and robust tail compression, each variate is patch-tokenized and projected into a unified latent space. The explicit handling of variate types—target, past covariate, and future-known covariate—is central: xLSTM processes targets and past-covariates in a strictly forward manner, upholding causality; only future-known covariates are processed bidirectionally, exploiting exogenous features without introducing future information leakage into targets.

Figure 2: Architectural schema—(Left) Bidirectional Time Mixer; (Middle) Alternation of time and variate mixing; (Right) Asymmetric grouped variate mixer yielding strict directionality and safe use of future-known covariates.

Asymmetric attention masking organizes variates into groups (one per multivariate input), enforcing that only target queries can attend to covariates, while covariate queries are forbidden from attending to targets. This precise masking is shown to be necessary and sufficient to guarantee strict target causality at all layer depths. The resulting representation allows safe, efficient use of future covariates for the forecast window, while enabling streaming state updates for targets and past covariates at constant cost.

Synthetic Coupling Pipeline for Data Generation

A key challenge for multivariate TSFM generalization is the lack of broad, high-quality multivariate corpora; large public datasets are principally univariate. TiRex-2 introduces a synthetic coupling pipeline that composes highly diverse multivariate samples from large univariate corpora with explicit, randomized cross-variate dependency structures: indirect mixing, structural causal models with lags and nonlinearities, cointegration, and deterministic functional relationships. The pipeline robustly replicates the range of regimes encountered in practical applications, including time warping, partial observability, and discretization.

Figure 3: Randomized synthetic pipeline: univariate series are augmented and coupled via cross-variate mechanisms (identity, SCMs, mixing, cointegration, coupling), followed by observational distortions and covariate enrichment.

This comprehensive coverage prevents inductive biases toward any single dependency pattern, offering the architectural backbone a broad substrate for learning genuinely universal cross-variate representations.

Zero-Shot and Streaming Forecasting Performance

TiRex-2 is evaluated against established TSFMs on two benchmarks: fev-bench (covariate exploitation, genuine multivariate dependencies) and GIFT-Eval (domain-diverse, frequency-agnostic, varied horizons). TiRex-2 outperforms all baselines in zero-shot accuracy on both point and probabilistic metrics, leading over Chronos-2 and Transformer-based methods despite a significantly lower active parameter budget.

Figure 4: fev-bench and GIFT-Eval zero-shot results: TiRex-2 achieves the lowest MASE and SQL/CRPS (higher accuracy, lower is better) in both benchmarks, outperforming larger models.

Figure 5: Pairwise win rate and skill score demonstrate dominant zero-shot performance for TiRex-2 on fev-bench (with 95% confidence intervals).

Notably, TiRex-2 is Pareto-optimal with respect to model size versus mean error—it achieves state-of-the-art accuracy with $\sim$2--4$\times$ fewer parameters than Chronos-2 or TimesFM (see also Figure 6).

Streaming evaluation confirms that the recurrent, memory-driven architecture maintains constant per-update computational cost with no degradation up to context lengths 4,000$\times$ longer than those observed during training. MASE remains flat well beyond the $8$k token post-training window.

Figure 7: Streaming MASE remains stable for contexts up to 32M steps, far beyond pre/post-training boundaries, underpinning robust online deployment.

Long-Horizon, Covariate Shift Sensitivity, and Ablations

Stress testing on chaotic synthetic benchmarks (dysts) demonstrates that TiRex-2 maintains lower MASE across forecast horizons and temporal granularities than competing TSFMs. Covariate-shift experiments where target and covariate lags are varied show TiRex-2 preserves useful predictive signal at lags where Chronos-2 and Transformer baselines lose all covariate benefit.

Ablation studies confirm that loss of grouped attention (cross-variate modeling), binary-tail compression in scaling, and bidirectional xLSTM severely degrade performance on genuinely multivariate/covariate tasks. The future-known covariate channel is necessary for leading accuracy; disabling it increases error notably, particularly on tasks with exogenous feature availability.

Parameter Efficiency and Scalability

A direct comparison of parameter count and mean benchmark error shows TiRex-2 forming the lower envelope of the state-of-the-art—no other model achieves superior accuracy at equal or lower parameter cost.

Figure 6: Pareto frontier on mean MASE vs. mean active parameters across nine zero-shot TSFMs—TiRex-2 is optimal for efficiency and accuracy.

Streaming evaluations on both univariate and multivariate variants reconfirm scalability: MASE is invariant over the range of cumulative streamed steps; context length is no limiting factor.

Figure 8: Streaming robustness: univariate and multivariate TiRex-2 maintain sublinear error as cumulative context increases.

Long-horizon forecasting experiments on chaotic dynamical systems (dysts) across resolutions (fine to coarse) highlight TiRex-2's dominance beyond short horizons.

Figure 9: MASE evolves with forecast horizon—the recurrent TiRex-2 backbone outperforms attention-based Chronos-2 across all granularities; advantage is pronounced at fine resolutions.

Theoretical and Practical Implications

By explicitly enforcing target causality and separating the mechanism of covariate integration, TiRex-2 resolves modeling artifacts that yield unreliable streaming deployment in prior TSFMs. The model's efficacy across dynamic, data-shifted, and covariate-misaligned regimes underlines its viability for real-world, high-stakes domains (e.g., disaster prediction, predictive maintenance, energy, and finance). Practically, TiRex-2's constant-memory, patch-wise streaming is essential for latency-sensitive online inference scenarios and for deployment on hardware-limited edge platforms.

On the theoretical front, the work exposes necessary and sufficient architectural constraints for safely leveraging exogenous, future-known information in autoregressive forecasting, preventing leakage, and enabling non-recombinatorial updating. The synthetic coupling generator conceptually closes the gap between univariate abundance and true multivariate structure, a major barrier for zero-shot generalization.

Conclusion

TiRex-2 demonstrates that recurrent architectures augmented with principled cross-variate attention can supersede Transformer-based TSFMs in both efficiency and accuracy for general zero-shot forecasting. It uniquely couples multivariate, streaming, and covariate-informed modeling in a single, unified framework with a modest parameter count. The empirical findings suggest a new baseline for TSFM design, with potential for future extensions including in-context adaptive covariate selection or further data-driven innovations within the coupling pipeline.

Ongoing limitations pertain to scenarios where future-known covariates change during the forecast window (requiring partial recomputation), though this is infrequent in standard practice. Streaming operates at patch granularity, aligning with the efficient tokenizer framework.

TiRex-2 is positioned to redefine architectures for universal time series foundation modeling, seamlessly integrating theoretical correctness, strong empirical accuracy, and real-world deployability.