Hierarchical SpatioTemporal Mamba (HiSTM)

• The paper presents HiSTM, a novel forecasting architecture integrating local 2D convolution, Mamba temporal modeling, and temporal attention to predict cellular traffic with notable accuracy gains.
• The model’s hierarchical encoder refines spatiotemporal features layer-by-layer, enabling robust single-step and multi-step forecasting over diverse urban grid configurations.
• Empirical results show HiSTM achieves up to a 47.3% MAE reduction and efficient performance using only 33,794 parameters, outperforming conventional models.

Hierarchical SpatioTemporal Mamba (HiSTM) is a spatiotemporal forecasting architecture for cellular traffic prediction that combines local spatial convolution, Mamba-based temporal sequence modeling, temporal attention, and a lightweight multilayer perceptron (MLP) head within a stacked encoder design (Bettouche et al., 7 Aug 2025). In its canonical formulation, HiSTM operates on local traffic patches extracted from a regular spatial grid, models the temporal evolution of each spatial position through selective state space methods, and then summarizes the center-cell trajectory for next-step prediction. Its defining characteristic is a layered spatiotemporal hierarchy: framewise spatial encoding is followed by temporal modeling inside each encoder layer, deeper layers progressively refine the representation, and a final attention stage converts patch-level context into target-cell prediction (Bettouche et al., 7 Aug 2025).

1. Problem formulation and domain assumptions

HiSTM is introduced for cellular traffic forecasting on a telecom-network grid, where the network state at time step $t$ is represented as a two-dimensional traffic map

$M_t \in \mathbb{R}^{H \times W}.$

Each cell stores a scalar traffic volume for a spatial region or base-station coverage area. The learning problem is localized around a target position $(i,j)$: the model takes a sequence of traffic patches

$X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$

and predicts the next-step traffic value at the center cell,

$x_{t+1}^{(i,j)} \in \mathbb{R}.$

The paper states the corresponding mapping as

$$f: \mathbb{R}^{T \times K \times K} \rightarrow \mathbb{R},$$

trained over many local windows (Bettouche et al., 7 Aug 2025).

The concrete experimental configuration uses $K=11$, $T=6$, one-step-ahead prediction of the center cell at the seventh time step, and multi-step autoregressive forecasting up to six steps. The datasets are represented on approximately $100 \times 100$ grids, and the model predicts each target cell from its local neighborhood rather than from the entire city-scale grid at once. This local-kernel formulation is a strong inductive bias: it assumes that the most useful spatial interactions for a cell are concentrated in its surrounding $11 \times 11$ region (Bettouche et al., 7 Aug 2025).

The motivation is the joint spatial-temporal difficulty of cellular traffic. The paper emphasizes that traffic is shaped by commuting patterns, overlapping coverage regions, user mobility across neighboring cells, local events, and heterogeneous urban, suburban, and rural profiles. It reports that a single-cell series has higher approximate entropy, $M_t \in \mathbb{R}^{H \times W}.$0, than aggregated traffic, $M_t \in \mathbb{R}^{H \times W}.$1, and that aggregated traffic shows stronger lag correlation and more visible cyclic structure. This is used to argue that purely univariate forecasting is inadequate, and that explicitly modeled spatial context can regularize volatile local signals (Bettouche et al., 7 Aug 2025).

2. Architectural composition and hierarchical structure

HiSTM takes an input patch sequence

$M_t \in \mathbb{R}^{H \times W}.$2

and first augments it with a channel dimension,

$M_t \in \mathbb{R}^{H \times W}.$3

The core encoder consists of $M_t \in \mathbb{R}^{H \times W}.$4 stacked layers. Each layer maps

$M_t \in \mathbb{R}^{H \times W}.$5

through a fixed sequence of operations: framewise 2D convolution with ReLU, reshaping so that each of the $M_t \in \mathbb{R}^{H \times W}.$6 spatial positions becomes a temporal sequence, Mamba sequence modeling along time for each position, and reshaping back into the spatiotemporal tensor. After $M_t \in \mathbb{R}^{H \times W}.$7 such layers, the encoded representation is

$M_t \in \mathbb{R}^{H \times W}.$8

The center-cell features across all time steps are then extracted,

$M_t \in \mathbb{R}^{H \times W}.$9

aggregated by temporal attention into a context vector $(i,j)$0, and passed to a two-layer MLP with ReLU to predict the next traffic value (Bettouche et al., 7 Aug 2025).

The paper defines the hierarchy at three levels. First, each encoder layer itself is hierarchical because it applies spatial convolution before temporal Mamba, so information flow is from raw patch to spatial features to temporal dynamics. Second, the stack of encoder layers forms a layer-wise feature hierarchy,

$(i,j)$1

with earlier layers handling more local or simple patterns and deeper layers representing more abstract spatiotemporal structure. Third, after patch-level encoding, the model introduces a final target-focused hierarchy by reducing patch-wide context to the center-cell trajectory and then to a scalar forecast through attention and MLP readout (Bettouche et al., 7 Aug 2025).

A noteworthy textual detail is that the abstract and conclusion describe a “dual spatial encoder,” whereas the methodology section concretely specifies stacked encoder layers, each composed of 2D convolution, temporal Mamba, and reshaping between the two. This makes the technical description of HiSTM more precise than the higher-level phrasing in the abstract (Bettouche et al., 7 Aug 2025).

3. Spatial modeling, temporal modeling, and formal equations

Spatial processing in HiSTM is implemented by a framewise convolutional encoder operating directly on regular grid patches. For encoder layer $(i,j)$2, the spatial transform is written as

$(i,j)$3

with output

$(i,j)$4

The tensor is then flattened for temporal modeling as

$(i,j)$5

Thus, convolution mixes neighboring cells within each frame, while Mamba models the evolution over time of each spatial position’s enriched features (Bettouche et al., 7 Aug 2025).

The temporal module uses standard Mamba/state-space dynamics rather than a HiSTM-specific recurrence. The paper gives the continuous-time and discrete-time state-space equations in the form

$(i,j)$6

and

$(i,j)$7

For selective Mamba-style parameterization, the conceptual form is

$(i,j)$8

leading to

$(i,j)$9

HiSTM sets $X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$0, so the channel dimension of the spatial encoder becomes the Mamba model dimension (Bettouche et al., 7 Aug 2025).

After spatiotemporal encoding, temporal attention is applied only to the center-cell sequence. With center-cell features $X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$1, the paper gives

$X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$2

$X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$3

$X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$4

This attention acts as an adaptive temporal readout over a Mamba-processed sequence rather than as a replacement for state-space recurrence (Bettouche et al., 7 Aug 2025).

A broader implication is that HiSTM uses a factorized spatiotemporal design: local spatial structure is extracted by convolution, temporal dependence is modeled by Mamba, and target-cell salience is imposed only at the final readout. In the surrounding Mamba literature, this is structurally different from flat token models such as VideoMamba, which treat video as a single spatiotemporal token sequence and apply Spatio-Temporal Forward and Backward SSM scans without an explicit local-patch center-cell forecasting formulation (Park et al., 2024).

4. Learning objective, preprocessing, and implementation profile

HiSTM is trained for single-step forecasting with Mean Absolute Error (MAE). The paper writes the loss as

$X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$5

Longer-horizon evaluation is performed autoregressively, feeding previous predictions back into the model to generate steps two through six (Bettouche et al., 7 Aug 2025).

The preprocessing pipeline is temporally conservative. Inputs and targets are scaled to $X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$6 using Min-Max normalization computed on training data only; validation and test sets reuse the same statistics, and out-of-range values are clipped. Temporal sampling uses a training stride of 6 and a test stride of 1. The chronological split is 70% train, 15% validation, and 15% test. Boundary effects are handled by cropping (Bettouche et al., 7 Aug 2025).

The implementation is in PyTorch and is trained on a single NVIDIA A100 80GB GPU with CUDA 12.4 and PyTorch 2.6.0+cu124. The reported optimization settings are batch size 128, maximum 40 epochs, early stopping patience 15, Adam optimizer, learning rate $X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$7, and ReduceLROnPlateau with patience 7 and factor 0.5 (Bettouche et al., 7 Aug 2025).

Efficiency is a major empirical theme. The model comparison table reports that HiSTM uses 33,794 parameters, has size 0.13 MB, GPU memory 10.63 MB, inference time 1.19 ms, and $X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$8 MACs. Relative to STN, this is presented as 94% fewer parameters; relative to VMRNN-D, about 18× fewer parameters; and relative to xLSTM, about 5.4× fewer parameters. The paper also notes that HiSTM’s MAC count is higher than STN’s, so its efficiency claim is not reducible to MAC minimization alone but is framed as an overall accuracy-efficiency tradeoff (Bettouche et al., 7 Aug 2025).

5. Empirical evaluation and observed performance

The experiments use the public dataset of Barlacchi et al., with Milan and Trentino sub-datasets. Milan contains 10,000 spatial cells with traffic recorded every 10 minutes; Trentino contains 11,466 spatial cells with the same temporal resolution but a different spatial density and activity profile. The baselines are STN, xLSTM, STTRE, VMRNN-B, and VMRNN-D, covering spatiotemporal CNN-style models, recurrent models, transformer-based models, and Mamba-based hybrids (Bettouche et al., 7 Aug 2025).

On single-step forecasting for Milan, HiSTM reports

• MAE 5.2196
• RMSE 11.2476
• $X_t^{(i,j)} \in \mathbb{R}^{T \times K \times K}$9
• SSIM 0.9925

against STN’s 7.3908 / 16.8824 / 0.9546 / 0.9853, STTRE’s 5.5558 / 11.4426 / 0.9791 / 0.9917, and competitive recurrent alternatives. The paper states a 29.4% MAE reduction over STN, a 2.3% SSIM gain over STN, and the best RMSE and $x_{t+1}^{(i,j)} \in \mathbb{R}.$0 among compared models (Bettouche et al., 7 Aug 2025).

For multi-step autoregressive forecasting on Milan, HiSTM remains best across all six forecast steps. Its MAE sequence is 3.87, 4.38, 4.85, 5.56, 6.09, and 6.69, compared with STN’s 5.28, 6.44, 7.39, 8.59, 9.55, and 10.59, and STTRE’s 4.21, 4.88, 5.47, 6.27, 6.88, and 7.54. At step 6, the paper highlights 36.8% lower MAE than STN and 11.3% lower MAE than STTRE, together with 58% slower error accumulation rate than STN. SSIM is also reported to decay more slowly, remaining above 0.95 at step 6 (Bettouche et al., 7 Aug 2025).

On cross-dataset generalization to Trentino, HiSTM reports

• MAE 1.3870
• RMSE 4.8134
• $x_{t+1}^{(i,j)} \in \mathbb{R}.$1
• SSIM 0.9916

which the paper summarizes as 47.3% MAE reduction over STN, 36.9% RMSE improvement over STN, and best SSIM and $x_{t+1}^{(i,j)} \in \mathbb{R}.$2. This cross-city result is used to support the claim that HiSTM generalizes across different spatial statistics and density profiles (Bettouche et al., 7 Aug 2025).

The paper also includes cell-specific analysis on a seven-day Milan test slice. Reported MAPE is 8.49% for an urban cell, 11.62% for a suburban cell, 24.86% for a rural cell, and 15.30% for a maximum-variance cell. The interpretation given is that performance is best in dense urban regions, stable in suburban settings, and worse in rural low-volume cells, while still remaining robust under high variability (Bettouche et al., 7 Aug 2025).

6. Position within the Mamba literature, limitations, and significance

HiSTM occupies a specific position within the emerging spatiotemporal Mamba family. Its hierarchy is local-grid and encoder-based rather than globally multiscale. This contrasts with VideoMamba, which is a flat, single-stage pure Mamba video backbone with tubelet tokenization and Spatio-Temporal Forward and Backward SSM scans but no multistage pyramid or patch-merging hierarchy (Park et al., 2024). It also contrasts with Mamba-Spike, whose serial spiking front-end, interface layer, and Mamba backbone constitute a stagewise hybrid pipeline that the supplied analysis explicitly characterizes as not a strong hierarchical spatiotemporal Mamba architecture (Qin et al., 2024).

Relative to more explicitly multiscale forecasting systems, HiSTM is also narrower in scope. STM3 uses parallel temporal Conv1D branches, multiscale Mamba with a scale-specific bias $x_{t+1}^{(i,j)} \in \mathbb{R}.$3, adaptive graph causal convolution, causal cross-scale attention, and mixture-of-experts routing; this is a much more explicit temporal hierarchy than HiSTM’s stacked local-patch encoder (Chen et al., 17 Aug 2025). In a different domain, HIGSTM for stock forecasting introduces commonality/specificity decomposition, temporal and global learned graphs, and information-guided selective SSMs, yielding another form of hierarchical spatiotemporal Mamba in which hierarchy is organized around market structure rather than grid-local patches (Yan et al., 14 Mar 2025).

The significance of HiSTM therefore lies less in proposing the most elaborate hierarchical Mamba formulation than in demonstrating a compact and effective local-grid CNN + temporal Mamba + attention design for telecom traffic forecasting. Its reported strengths are best single-step performance on Milan, strongest cross-dataset generalization on Trentino, strong multi-step stability, very low parameter count, and practical inference time (Bettouche et al., 7 Aug 2025).

Its limitations are equally explicit. The paper notes that broader validation on more diverse geographies is still needed; missing-data handling is not addressed; concept drift adaptation remains open; spatial heterogeneity could be handled better with adaptive weighting; attention visualization is left for future work; rural and sparse regions remain harder; and high peaks may still be underestimated (Bettouche et al., 7 Aug 2025). A plausible implication is that HiSTM is best viewed as a compact domain-specific realization of hierarchical spatiotemporal Mamba principles rather than as a universal template for all hierarchical Mamba architectures.