--- title: Multi-Scale Structured State-Space Mixtures (MS3M) url: https://www.emergentmind.com/topics/multi-scale-structured-state-space-mixtures-ms3m type: topic --- # Multi-Scale Structured State-Space Mixtures (MS3M) Searching arXiv for MS3M and related multiscale structured state-space model papers. Searching arXiv for "Multi-Scale Structured State-Space Mixtures" and related titles. Multi-Scale Structured State-Space Mixtures (MS3M) are compact, strictly causal, multi-layer structured state-space forecasters introduced for near-real-time analytics in sixth-generation Open Radio Access Networks (6G O-RAN). In the reported formulation, each layer mixes multiple HiPPO-LegS state-space kernels with learnable time constants to capture fast and slow temporal dynamics, applies their causal impulse responses as per-feature depthwise convolutions, and augments the resulting linear memory mechanism with Squeeze-and-Excitation gating and a compact gated channel-mixing layer for nonlinear cross-feature interactions [2510.05255]. A later extension, WM-MS3M, couples this backbone with a compact stochastic latent to support action-conditioned forecasting, uncertainty-aware rollouts, and counterfactual planning in Near-RT control loops [2511.02748]. The acronym “MS3M” is not uniform across the literature: it is also used for a different multi-scale state space model in event-camera deraining [2505.05307]. ## 1. Definition and conceptual scope In its O-RAN formulation, MS3M is a lightweight forecaster for noisy, multivariate radio telemetry, trained on sliding windows to predict the immediate next step. The model is described as KPI-agnostic; Reference Signal Received Power (RSRP) is used as a canonical use case, but the same backbone is intended for short-horizon per-User Equipment key performance indicator forecasting more generally [2510.05255]. The defining idea is the combination of structured state-space memory with explicit multi-timescale mixing. Rather than relying on a single recurrence or self-attention mechanism, MS3M instantiates several state-space kernels per layer and sums them: $$ k^{(\ell)}[\cdot] = \sum_{m=1}^M k^{(\ell,m)}[\cdot]. $$ This “mixture” is therefore a mixture of kernelized temporal dynamics. It is not presented as a switching-regime model with discrete latent selectors, nor as a generic label for all multiscale SSMs. That distinction matters because other multiscale state-space literatures use “mixture” to denote different mechanisms, including regime switching and latent scale-mixture priors. Within the supplied literature, MS3M occupies a specific position among efficient sequence models: it is designed to retain causal temporal modeling and stable fading memory while reducing latency and data movement relative to Transformer baselines under Near-RT constraints. This focus makes the term narrower than the broader category of multiscale SSMs in vision, time-series, or Bayesian dynamical systems. ## 2. Architectural organization of the original MS3M forecaster The reported architecture comprises an input embedding, a stack of multi-scale state-space layers, Squeeze-and-Excitation (SE) gating, a gated linear unit (GLU)-style channel mixer, and a prediction head that emits next-step forecasts from the most recent latent representation [2510.05255]. The core temporal operator is a strictly causal depthwise convolution whose filter coefficients are generated from discretized structured state-space models. For layer $\ell$ and channel $c$, the convolution is $$ U^{(\ell)}_{t,c} = \sum_{\tau=0}^{L_k-1} k^{(\ell)}_c[\tau]\; H^{(\ell-1)}_{t-\tau,c}, $$ with left zero-padding to preserve strict causality. Each channel is therefore filtered independently by a structured temporal kernel, after which channelwise and cross-channel modulation are introduced by the nonlinear blocks. The SE gate summarizes the latent sequence across time, $$ s^{(\ell)} = \frac{1}{L}\sum_{t=1}^{L} H^{(\ell-1)}_{t,\cdot}, $$ then generates channel weights $$ g^{(\ell)} = \sigma\!\left(W_2^{(\ell)}\phi\!\left(W_1^{(\ell)}s^{(\ell)}\right)\right), $$ which rescale the depthwise-convolution output: $$ \widehat{H}^{(\ell)}_{t,\cdot} = U^{(\ell)}_{t,\cdot}\odot g^{(\ell)}. $$ A residual update and layer normalization produce $$ Y^{(\ell)} = \mathrm{LN}\!\left(H^{(\ell-1)} + \widehat{H}^{(\ell)}\right). $$ Cross-feature nonlinearities are then handled by a compact GLU mixer, $$ Z^{(\ell)} = W_{\downarrow}^{(\ell)} \left[ \phi\!\left(W_{\uparrow,a}^{(\ell)}Y^{(\ell)}\right) \odot \sigma\!\left(W_{\uparrow,g}^{(\ell)}Y^{(\ell)}\right) \right], $$ followed by $$ H^{(\ell)} = \mathrm{LN}\!\left(Y^{(\ell)} + Z^{(\ell)}\right). $$ This division of labor is central to the model definition. The structured SSM kernels handle long-memory temporal filtering; SE gating dynamically reweights KPI channels as conditions change; and the GLU mixer introduces nonlinear cross-channel interactions without Transformer-level cost. The output head uses only the most recent time step: $$ \hat{y} = W_{\text{head}} H^{(L_\ell)}_{L,\cdot} + b_{\text{head}}. $$ ## 3. HiPPO-LegS kernels, bilinear discretization, and stability MS3M’s structured kernels are derived from HiPPO-LegS continuous-time operators. In the reported formulation, the continuous-time transition matrix and reference input vector are $$ (A_{ct})_{ij}= \begin{cases} -\sqrt{(2i+1)(2j+1)}, & i>j\\ -(i+1), & i=j\\ 0, & i0$: $$ \rho(A(\Delta t))<1. $$ This stability-by-construction property is one of the reasons MS3M is positioned as suitable for long-memory forecasting under operational constraints [2510.05255]. The multi-scale aspect enters through learnable step sizes or time constants across mixture components. Each component contributes a kernel with a distinct temporal footprint, and their sum forms the effective filter for a layer. This yields a compact representation of multi-timescale dynamics without reverting to quadratic self-attention. In the O-RAN use case, the intended target is the coexistence of bursty and slowly varying radio processes. ## 4. Training protocol, reported forecasting results, and operational profile The original MS3M forecaster is trained on sliding windows of standardized KPI sequences to predict the immediate next step. The reported pipeline emphasizes strict causality and leakage prevention: preprocessing, standardization, and data splits are fitted on the training set, with no test-time lookahead. The optimization objective is standard mean squared error in standardized space, with weight decay and early stopping via validation loss [2510.05255]. Empirical evaluation is reported on a bespoke O-RAN testbed KPI time-series dataset consisting of 59,441 windows across 13 KPIs. The primary benchmark is one-step-ahead RSRP prediction, though the architecture is described as KPI-agnostic. On the reported hardware and setup, MS3M attains a 0.057 s per-inference latency with 0.70M parameters, and the paper states that this yields 3–10x lower latency than the Transformer baselines evaluated on the same hardware while maintaining competitive accuracy [2510.05255]. The detailed comparison table places MS3M at 698,449 parameters, 0.057 s latency, 0.292 RMSE, and 0.170 MAE. In that table, Crossformer achieves the lowest reported RMSE at 0.275 and MAE at 0.154, but with 1,591,321 parameters and 0.586 s latency; Informer, TFT, ETSformer, PatchTST, and iTransformer are all slower than MS3M, with higher or much higher reported error depending on the model [2510.05255]. The operational significance is explicit: the model is intended to preserve forecasting quality while reducing latency and data movement enough for Near-RT deployment. A recurrent point in the evaluation is that efficiency is not incidental to the architecture; it is the design objective. Per-feature depthwise SSM filtering, compact gating, and lightweight channel mixing are all chosen to avoid Transformer-level memory and latency overhead while remaining strictly causal. ## 5. WM-MS3M: action-conditioned world modeling and counterfactual forecasting WM-MS3M extends the causal MS3M backbone by adding a compact stochastic latent and dual decoders, reframing next-step KPI forecasting as part of a world-modeling pipeline for 6G O-RAN control [2511.02748]. The model takes causal windows of KPI histories and action histories, where actions are physical resource block allocations, and predicts next-step KPIs under either factual or hypothetical action sequences. The conditioning interface is $$ f_\theta:(X_t, u_{t-L+1:t})\mapsto \widehat{y}_{t+1}, $$ with $$ X_t=(x_{t-L+1},\dots,x_t)\in\mathbb{R}^{L\times F}. $$ The extension introduces a diagonal-Gaussian latent $z$ with a learned prior and posterior, $$ p_\theta(z|d)=\mathcal{N}(\mu_p,\operatorname{diag}(\sigma_p^2)), \qquad q_\theta(z|d,\tilde{y}_{\text{full}})= \mathcal{N}(\mu_q,\operatorname{diag}(\sigma_q^2)), $$ where the deterministic summary $d$ is produced by the MS3M backbone. Decoding proceeds through a full multivariate reconstruction head and a target heteroscedastic head, $$ \hat{y}_{\text{full}}=\mathrm{Dec}_{\text{full}}([d;z]), $$ $$ (\mu_t,\log \sigma_t^2)=\mathrm{Dec}_{t}([d;z]). $$ A bounded autoregressive skip further sharpens short-horizon forecasts: $$ p_\theta(\tilde{y}^{(O)}|d,z,\tilde{x})= \mathcal{N}(\mu_t+\alpha b,\operatorname{diag}(\sigma_t^2)), \qquad b=f_{\mathrm{ar}}(\tilde{x}_t),\;\alpha=\tanh(\kappa). $$ Training uses a KL-annealed objective with full-head reconstruction, target-head negative log-likelihood, Huber auxiliary loss, consistency loss, KL regularization, and an optional teacher-forced multi-step rollout penalty: $$ \mathcal{L}= \alpha_{\mathrm{rec}}\mathcal{L}_{\mathrm{recon}}+ \alpha_{\mathrm{t}}\mathcal{L}_{\mathrm{t}}+ \alpha_{\mathrm{hub}}\mathcal{L}_{\mathrm{hub}}+ \alpha_{\mathrm{cons}}\mathcal{L}_{\mathrm{cons}}+ \beta_e\mathcal{L}_{\mathrm{kl}}+ \alpha_{\mathrm{roll}}\mathcal{L}_{\mathrm{roll}}. $$ At inference time, the model samples from the prior, averages over Monte Carlo rollouts, and reverses standardization to obtain calibrated means and uncertainties. The world-model role is operationalized through an MPC-based cross-entropy method planner that rolls out short-horizon futures under candidate PRB sequences and scores them with a deterministic reward. The reported “what-if” scenarios include Hold, Step +20%, Step -20%, Ramp to high, and MPC/CEM policy. In the example table, Step -20% PRB achieves the best total reward at +1.066, while Hold gives -0.232 and Step +20% gives -1.585 [2511.02748]. In the reported test-set comparison, WM-MS3M attains RMSE 0.2917, MAE 0.1674, $R^2=0.9930$, 477,802 parameters, and 0.00065 s inference, while deterministic MS3M is reported at RMSE 0.2918, MAE 0.1703, 698,449 parameters, and 0.00064 s inference. The paper states that WM-MS3M cuts MAE by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35–80% lower RMSE than attention/hybrid baselines with 2.3–4.1x faster inference [2511.02748]. In this form, MS3M becomes not only a forecaster but also the backbone of a causal, uncertainty-aware simulator for offline policy screening and rare-event analysis. ## 6. Relation to adjacent multiscale SSMs and recurring misconceptions Several recent models share the multiscale SSM design motif without being the same architecture. The distinction is easiest to see by domain and mechanism. | Model | Domain | Distinguishing mechanism | |---|---|---| | MS3M [2510.05255] | 6G O-RAN KPI forecasting | HiPPO-LegS kernel mixtures, depthwise causal convolution, SE, GLU | | WM-MS3M [2511.02748] | O-RAN world modeling | MS3M backbone plus stochastic latent, dual decoders, MPC/CEM rollouts | | MS-SSM [2512.23824] | General sequence modeling | Multi-scale decomposition block, parallel SSMs, input-dependent scale-mixer | | Serpent [2403.17902] | Image restoration | Multi-directional visual SSM blocks in a U-Net-like hierarchy | | MSVMamba [2405.14174] | Vision backbones | Multi-scale 2D scanning on full and downsampled feature maps | | PRE-Mamba “MS3M” [2505.05307] | Event-camera deraining | Dual-temporal and multi-spatial branches with Mamba core | MS-SSM processes multiple learned resolutions in parallel and recombines them with an input-dependent scale-mixer; it is reported to outperform prior SSM-based models on Long Range Arena, hierarchical reasoning, time-series classification, and image recognition, including a mean mixing distance of 94.9 versus 38.8 for Mamba on ListOps [2512.23824]. Serpent applies selective SSMs to multi-directionally unrolled image patches inside a hierarchical restoration network and reports up to 150-fold reduction in FLOPS and up to $5\times$ less GPU memory while maintaining compact model size [2403.17902]. MSVMamba reduces scan redundancy by combining one full-resolution scan with multiple downsampled scans, and the reported MSVMamba-Tiny achieves 82.8% top-1 accuracy on ImageNet with 17% lower FLOPs than VMamba-Tiny [2405.14174]. PRE-Mamba reuses the acronym “MS3M” for a Multi-Scale State Space Model in event-camera deraining, where it captures deeper rain dynamics across dual-temporal and multi-spatial scales with linear computational complexity and is reported together with 0.95 SR, 0.91 NR, 0.4s/M events, and 0.26M parameters on EventRain-27K [2505.05307]. A common misconception is to treat all of these models as interchangeable simply because they are multiscale and SSM-based. The supplied literature does not support that equivalence. In the O-RAN MS3M, “mixture” denotes the sum of several HiPPO-LegS kernel components with learnable time constants inside each layer [2510.05255]. By contrast, in a generalized multiscale Bayesian framework with switching regimes, mixture-like behavior is induced by discrete latent indicators $s_{d,t_L}^{[L]}$ and Dirichlet-distributed regime probabilities, with inference performed by multiscale Sequential Monte Carlo rather than causal depthwise convolution [2410.19074]. Earlier multiscale probabilistic work also modeled hierarchical dependencies through latent tree-structured scale mixture priors and scalable variational Bayes, especially for imaging problems [1206.6437]. This suggests a broader prehistory of multiscale dependency modeling, but not an identity of formulations. Another misconception is that general SSM surveys necessarily cover MS3M. In the supplied material, the record for “From S4 to Mamba: A Comprehensive Survey on Structured State Space Models” is explicitly described as not containing any discussion or categorization of SSMs and as not mentioning multi-scale, mixture-based, or MS3M approaches [2503.18970]. The term therefore remains more specific and less standardized than labels such as S4, Mamba, or S6. In summary, MS3M designates a particular family of causal multiscale structured-state-space forecasters centered on kernel mixtures, stable discretization, depthwise temporal filtering, and lightweight nonlinear gating. Its most developed formulation in the supplied literature is the O-RAN forecasting and world-modeling line, where accuracy, latency, strict causality, and counterfactual action conditioning are treated as coequal requirements rather than separate objectives. Source: https://www.emergentmind.com/topics/multi-scale-structured-state-space-mixtures-ms3m