xLSTMTime: Advanced Time Series Modeling

• xLSTMTime is a family of advanced recurrent neural architectures that use exponential gating and expanded memory to capture long-term temporal dependencies.
• It introduces scalar, matrix, and time-aware variants to mitigate vanishing gradients and enhance accuracy in multivariate forecasting tasks.
• Its stochastic extensions and optimized training strategies enable robust uncertainty quantification and efficient deployment in business, weather, and traffic domains.

xLSTMTime refers to a family of advanced recurrent neural network architectures developed for time-aware and long-term modeling of temporal sequences, especially in scenarios where conventional LSTM variants are limited by vanishing gradients, poor temporal credit assignment, or inability to encode uncertainty. The term encompasses models that build upon the foundational xLSTM concept—introducing exponentially parameterized gating, expanded memory mechanisms, and (in recent variants) stochastic state-space extensions. xLSTMTime architectures have been key in multivariate long-term time series forecasting, predictive business process monitoring, and spatiotemporal forecasting, offering improved performance and efficiency over Transformer-based and traditional linear baselines.

1. Architectural Innovations in xLSTMTime

xLSTMTime models originate from the extended LSTM (xLSTM) framework, whose primary enhancements include replacing standard sigmoid gates with exponential gating functions and employing revised memory structures to better capture long-term dependencies (Alharthi et al., 2024, Wang et al., 1 Sep 2025). Several design variants are used, most notably:

• Scalar-LSTM (sLSTM): All gates (input, forget, output) are computed as a single scalar shared across channels, reducing parameter count and improving gradient flow. The gating equations for sLSTM replace $\sigma(\cdot)$ activations with exponentials, which mitigates saturation and enables finer temporal sensitivity over long sequences.
• Matrix-LSTM (mLSTM): The memory cell is promoted from vector to matrix, with updates leveraging outer products of "key" and "value" projections, allowing the architecture to encode richer, multi-dimensional memory representations. This design omits hidden-to-hidden recurrences in gate computation, supporting sequence-parallel GPU execution.
• Time-aware LSTM (T-LSTM): In settings with irregularly sampled event logs (e.g., business processes), T-LSTM decomposes the previous memory into short- and long-term components, applying a monotonic time decay to the former based on the inter-event interval $\Delta t$ (Nguyen et al., 2020).

The following table outlines key memory and gating mechanisms in the main xLSTMTime cell designs:

Variant	Gating Function	Memory Structure
sLSTM	Exponential	Scalar vector
mLSTM	Exponential	Memory matrix
T-LSTM	Sigmoid + decay	Decayed subspaces

2. Exponential Gating and Memory Mechanisms

In xLSTMTime architectures, exponential gating replaces the standard sigmoid activations ($\sigma(\cdot)$), resulting in input and forget gates defined as $i_t = \exp(W_i x_t + R_i h_{t-1} + b_i)$ and $f_t = \exp(W_f x_t + R_f h_{t-1} + b_f)$. These are numerically stabilized with a running log-max variable $m_t$:

$$m_t = \max(\log f_t + m_{t-1},\; \log i_t), \quad i_t' = \exp(\log i_t - m_t), \quad f_t' = \exp(\log f_t + m_{t-1} - m_t)$$

This formulation prevents saturation, supports long-range gradient propagation, and enables xLSTMTime to outperform standard LSTM in long-sequence or data-sparse regimes. In mLSTM, outer product memory updates ($i_t V_t K_t^\top$) further extend the model’s storage and association capacity. Empirical ablation indicates that removing exponential gating degrades long-horizon accuracy by 8–12%, and substituting mLSTM with standard LSTM on large datasets worsens error by 10–15% (Alharthi et al., 2024).

3. Temporal Modeling with Time-Aware and Stochastic Extensions

xLSTMTime models have been adapted for specialized temporal contexts:

• T-LSTM for Irregular Event Sequences: T-LSTM distinguishes short-term ($c_S$) and long-term ($c_T$) memory within each cell state, explicitly discounting $c_S$ by a decay function of inter-event elapsed time $\Delta t$, e.g., $\gamma^{(t)} = 1/\log(e+\Delta t^{(t)})$, to adjust the cell’s historical influence (Nguyen et al., 2020).
• StoxLSTM for Uncertainty Modeling: StoxLSTM extends xLSTM by embedding a stochastic latent variable $z_t$ at each time step, transforming the architecture into a deep state-space model. The generative process factorizes over observed $x_{t}$ and latent $z_t$ variables, with inference via bidirectional variational approximation. The ELBO objective combines reconstruction (MSE) and closed-form KL terms, yielding forecasts with uncertainty quantification and improved robustness in highly stochastic environments (Wang et al., 1 Sep 2025).

4. Training Procedures, Hyperparameters, and Regularization

xLSTMTime models are optimized using standard routines (Adam, RAdam, Nadam) with loss functions tailored to the application—MAE for long-term forecasting, weighted cross-entropy for imbalanced classification, and composite objectives in multitask settings. Key regularization strategies include batch normalization in both feature and output spaces, early stopping, dropout in dense layers, and (for StoxLSTM) patching with channel independence to reduce overfitting.

Common hyperparameter choices are:

• Learning rate: $1e^{-3}$ (Adam, sLSTM/mLSTM); $[1e^{-4}, 1e^{-2}]$ (Nadam/T-LSTM)
• Batch size: 32 or 64
• Dropout: 0.0–0.2 (as needed)
• Patience: 25 epochs (early stopping)
• Memory dim: 64–100 (sLSTM/T-LSTM); latent size 16 (StoxLSTM)

Empirical training times are competitive: xLSTMTime often scales linearly with sequence length and achieves 2–4x faster inference than O($L^2$)-complexity Transformer baselines for long horizons (Alharthi et al., 2024).

5. Empirical Evaluation and Benchmarking

xLSTMTime has demonstrated state-of-the-art or near-best forecasting metrics across diverse time series tasks:

• Long-term Multivariate Forecasting: On 12 real-world datasets (ETTh1/2, ETTm1/2, Electricity, Traffic, Weather, ILI, PeMS03/04/07/08), xLSTMTime achieves the top MSE/MAE in 9/12 cases (T=96), with up to 18% MAE reduction vs. DLinear and consistent improvements over PatchTST for long horizons (Alharthi et al., 2024).
• Business Process Monitoring: T-LSTM with cost-sensitive learning improves next-activity classification accuracy by +1.2–1.8 pp and reduces next-timestamp MAE by up to 0.9 days over vanilla LSTM on Helpdesk and BPI12W logs (Nguyen et al., 2020).
• Stochastic and Hierarchical Dynamics: StoxLSTM outperforms xLSTM and state-of-the-art Transformer and linear models on long-term forecasting tasks, winning 33/44 MSE/MAE slots and achieving substantial improvements in MAE, MAPE, and RMSE on short-horizon benchmarks (Wang et al., 1 Sep 2025).
• Spatiotemporal Traffic Forecasting: In cellular networks, xLSTMTime (STN-sLSTM-TF) achieves a 23% test MAE reduction over ConvLSTM and up to 36% improvement in MAE generalization to held-out regions—enabled by scalar gating and cross-attention integration (Ali et al., 17 Jul 2025).

6. Application Domains and Practical Considerations

xLSTMTime has been applied to:

• Multivariate and univariate time series forecasting (energy, weather, epidemiology, traffic)
• Predictive business process monitoring, including next-activity and next-timestamp tasks
• Spatiotemporal modeling in cellular traffic forecasting, outperforming ConvLSTM and standard STN variants

Training typically requires commodity GPUs (e.g., RTX 3090, A800, Quadro RTX6000), with single runs completing within minutes for moderate sequences. T-LSTM and xLSTMTime cells are drop-in compatible with existing LSTM code. The memory, parameter, and inference footprint is competitive with or better than transformer models—scaling linearly in sequence length and supporting efficient deployment (Alharthi et al., 2024, Nguyen et al., 2020, Ali et al., 17 Jul 2025).

7. Theoretical and Empirical Insights

xLSTMTime’s performance gains are attributed to several core mechanisms:

• Exponential gating avoids the vanishing-gradient pathology of sigmoid gates, preserving memory relevance over hundreds of steps.
• High-capacity matrix memory (in mLSTM) allows the model to encode richer temporal associations and hierarchical patterns.
• Time-aware cell modifications (T-LSTM) are essential for modeling irregular event sequences and aligning temporal influence with event intervals.
• Stochastic latent augmentation (StoxLSTM) enables the model to capture uncertainty, multimodality, and hierarchical state-space structure, which deterministic recurrent architectures cannot represent.

A plausible implication is that as time series complexity increases—due to longer dependencies, uncertainty, or spatiotemporal context—xLSTMTime’s architectural innovations offer a robust and efficient alternative to both transformer and shallow linear models, while facilitating end-to-end training and practical deployment across domains (Wang et al., 1 Sep 2025, Alharthi et al., 2024, Ali et al., 17 Jul 2025, Nguyen et al., 2020).