xLSTMAD-F Anomaly Detection Framework

• xLSTMAD-F is a forecasting-based anomaly detection framework that employs an encoder–decoder design with xLSTM cells featuring exponential gating and residual connections.
• It uses iterative autoregressive decoding to forecast future time steps and compare predictions with actual observations for effective anomaly scoring.
• Experimental evaluations show that xLSTMAD-F outperforms baselines, achieving superior AUC-PR and AUC-ROC scores on real-world multivariate time series datasets.

xLSTMAD-F is a forecasting-based anomaly detection framework that builds on the extended Long Short-Term Memory (xLSTM) architecture. It is designed for multivariate time series anomaly detection, leveraging an encoder–decoder model built entirely from xLSTM blocks. By employing multiplicative (exponential) gating, residual connections, and iterative autoregressive decoding, xLSTMAD-F achieves superior detection accuracy compared to established baselines on real-world datasets.

1. Architectural Principles

xLSTMAD-F utilizes an encoder–decoder design in which both components are constructed from xLSTM units. The encoder processes a windowed sequence of multivariate time series data to produce hidden representations that encapsulate historical context. The decoder iteratively forecasts p future time steps based on this context, forming predictions that are then compared to actual observations for anomaly scoring.

A central feature of the architecture is the xLSTM cell, which extends traditional LSTM mechanisms in several important ways:

• Exponential Gating: Input and forget gates use the exponential function (e.g., $i_t = \exp(\tilde{i_t}),\ f_t = \exp(\tilde{f_t})$), enabling more expressive and sharp weighting of new versus historical information.
• Normalizer State: To maintain numerical stability amidst the wide dynamic range made possible by exponential gating, a normalizer variable accumulates gate activations and provides division normalization.
• Residual Connections: Stacked xLSTM blocks are equipped with skip connections (e.g., $H_\ell = H_{\ell-1} + H_\ell^{(f)}$), promoting stable gradient flow and improved training of deep sequence models.
• 1D Convolutions: Prior to xLSTM processing, some blocks employ 1D convolutional layers (kernel size 4 or 8) to better extract local temporal features and facilitate representation learning across both local and global patterns.

The forecasting variant, xLSTMAD-F, contrasts with the reconstruction variant (xLSTMAD-R) by iteratively generating forecasts rather than reconstructing past inputs.

2. Methodology and Sequence Modeling

xLSTMAD-F operates on an autoregressive forecast-based paradigm:

1. Encoding: The encoder consumes a historical input window $X_t^{(W)}$ and, via a sequence of xLSTM blocks, produces a sequence of hidden states. The last time-step’s hidden state $h_0$ initializes the decoder.
2. Decoding (Forecasting): For each of the $\tau$ forecast steps, the decoder recursively updates its hidden state using the xLSTM cell:

$h_t = \mathrm{xLSTM_{dec}}(h_{t-1})$

The predicted output at each step is obtained through a nonlinearity (e.g., GELU) and a linear projection:

$\hat{y}_t = \varphi(z_t W_o + b_o)$

where $z_t$ is the processed decoder state.

1. Loss and Anomaly Scoring: Prediction errors across the future window are aggregated via a loss metric (see next section). Elevated error indicates points of anomalous behavior.

A key formula for the pointwise prediction loss is:

$$L_{\mathrm{pred}}(t) = \frac{1}{p \cdot D} \sum_{i=1}^p \sum_{j=1}^D \left( x_{t+i, j} - \hat{y}_{t+i, j} \right)^2$$

where $p$ is the forecast length and $D$ the number of features.

3. Loss Functions for Anomaly Detection

Two principal loss functions are employed within xLSTMAD-F:

• Mean Squared Error (MSE): Measures pointwise deviation between predicted and ground truth values; emphasizes local fidelity and detects sharp anomalies.
• Soft Dynamic Time Warping (SoftDTW): Provides a differentiable approximation to DTW, accommodating temporal warping or phase shifts in anomalous sequences. The SoftDTW loss is defined recursively:

$C_{i,j} = \Vert X_i - \hat{Y}_j \Vert^2$

$$D_\gamma(i,j) = C_{i,j} + \gamma \cdot \mathrm{softmin}\big(D_\gamma(i-1, j), D_\gamma(i, j-1), D_\gamma(i-1, j-1)\big)$$

where $\gamma$ is a smoothing parameter. The final loss is $D_\gamma(T, T')$.

The use of SoftDTW enables more robust alignment and scoring in the presence of temporal distortions frequently observed in time series anomalies.

4. Experimental Evaluation and Benchmarking

xLSTMAD-F was evaluated extensively on the TSB-AD-M benchmark, which comprises 17 real-world multivariate time series datasets. Comparative analysis involved 23 state-of-the-art baselines—including various LSTM-based, CNN, and Transformer anomaly detection models.

Performance was measured with several advanced metrics:

• VUS-PR: Volume Under the Surface for the Precision-Recall curve, assessing a model’s prioritization of true anomalies.
• AUC-PR and AUC-ROC: Standard threshold-agnostic classification measures.
• F1-Score Variants: For precision-recall-focused assessment.

Results demonstrated that xLSTMAD-F, particularly when trained with the MSE loss, achieved an AUC-PR of 0.35 and an AUC-ROC of 0.74, surpassing all baselines in detecting both abrupt and subtly evolving anomalies. The iterative forecast approach facilitated modeling of long-range dependencies, providing both predictive power and fine-grained anomaly localization.

5. Significance and Applications

xLSTMAD-F showcases the effectiveness of modern recurrent sequence models—specifically those incorporating exponential gating, normalizer states, and residual stacking—for anomaly detection in complex, multivariate time series.

Key implications for practice include:

• Robust Detection: The method balances historical context modeling and forward prediction, capturing both short-term and long-term dependencies critical for effective anomaly detection.
• Flexible Loss Choices: The option to use both local (MSE) and global (SoftDTW) sequence alignment losses allows tailoring to applications with different anomaly characteristics.
• Efficient Representation: Enhanced temporal feature extraction via convolutions and deep stacking enables detection in large-scale or high-frequency data without the quadratic memory or inference cost of attention-based models.

Potential application domains include industrial process monitoring, fraud or network intrusion detection, medical physiology analytics, and adaptive behavior analysis in complex systems. A plausible implication is that the residual and exponential-gated xLSTM backbone can be further extended to other sequential modeling domains beyond anomaly detection, such as online forecasting and representation learning for time series.

6. Future Directions and Variants

The xLSTMAD framework supports both forecasting-based (xLSTMAD-F) and reconstruction-based (xLSTMAD-R) anomaly detection, opening research avenues in hybrid or task-specific approaches. Future work may explore:

• Optimizing Forecast vs. Reconstruction Trade-offs: Determining the best strategy for specific anomaly types or data domains.
• Scaling to Longer Sequences: Adapting the architecture and training regimes to handle ultra-long sequences efficiently.
• Transfer and Pretraining: Leveraging pre-trained xLSTMAD-F backbones for domain adaptation or multi-task anomaly detection.

The design also allows for extension to alternative deep learning toolkits and distributed settings, given the architecture’s computational efficiency and modularity. As xLSTMAD-F establishes new benchmarks for detection accuracy and efficiency, continued exploration of its design principles is expected within the time series, anomaly detection, and recurrent modeling communities (2506.22837).