S-Mamba: Specialized Mamba Models

• S-Mamba is a family of specialized Mamba variants that apply selective state-space modeling to efficiently capture inter-variate correlations and domain structures.
• The models use a hybrid design, combining bidirectional Mamba layers for cross-variate encoding with feed-forward networks for temporal dependencies.
• Empirical studies show that S-Mamba variants achieve strong performance with lower computational costs, especially on high-variate or periodic datasets.

Searching arXiv for the most relevant papers on “S-Mamba” and closely related exact-name variants. In the arXiv literature, S-Mamba is not a single universally fixed designation but a label used for several Mamba-family architectures, most explicitly “Simple-Mamba (S-Mamba)” for multivariate time-series forecasting and “Sparse-Mamba (S-Mamba)” for control-structured natural-language state space models (Wang et al., 2024, Hamdan et al., 2024). In its most established exact-name use, S-Mamba denotes a deliberately lightweight Mamba-based model for multivariate long-term time-series forecasting that tokenizes each variate independently, uses a bidirectional Mamba layer to encode inter-variate correlation, and uses a feed-forward network to encode temporal dependency (Wang et al., 2024). More broadly, the term also appears in adjacent or easily confusable forms such as SS-Mamba for hyperspectral image classification (Huang et al., 2024), S$^2$Mamba for spatial-spectral hyperspectral modeling (Wang et al., 2024), and Speech-Mamba as a speech-oriented selective state space model for long-context automatic speech recognition (Gao et al., 2024). This multiplicity makes “S-Mamba” best understood as a family label whose meaning depends on domain and paper context.

1. Terminological scope and principal referents

The most direct exact-name use of S-Mamba in the provided literature is “Simple-Mamba”, introduced for multivariate time-series forecasting (TSF) (Wang et al., 2024). That paper studies the standard long-term forecasting problem, where a model observes a historical window

$$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$

and predicts

$$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$

Its defining claim is that Mamba is especially effective for inter-variate correlation encoding, while temporal modeling can remain structurally simple (Wang et al., 2024).

A second exact-name use is “Sparse-Mamba (S-Mamba)”, which introduces controllability, observability, and stability into Mamba-style structured state space models for natural language processing applications (Hamdan et al., 2024). That work defines a family containing SC-Mamba, SO-Mamba, and ST-Mamba2, and its notion of sparsity refers to canonical control-theoretic parameterization of the state matrix $A$, not pruning or sparse activations (Hamdan et al., 2024).

The broader Mamba literature also contains near matches that are frequently relevant when the string “S-Mamba” is used informally. In hyperspectral image classification, SS-Mamba denotes “Spectral-Spatial Mamba”, a dual-branch HSI classifier with spectral and spatial token streams (Huang et al., 2024). A closely related but distinct model is S$^2$Mamba, a spatial-spectral state space model that applies selective SSMs along spatial and spectral dimensions and fuses them with a Spatial-spectral Mixture Gate (Wang et al., 2024). In speech technology, Speech-Mamba is explicitly described as a reasonable instance of “S-Mamba” in the sense of a speech-oriented selective state space model for long-context ASR (Gao et al., 2024).

This usage pattern suggests that “S-Mamba” functions less as a canonical architecture name than as a recurrent naming convention for specialized Mamba variants. In most contexts, however, the exact unqualified form S-Mamba most naturally points to Simple-Mamba (Wang et al., 2024) or Sparse-Mamba (Hamdan et al., 2024).

2. Simple-Mamba for time-series forecasting

In Simple-Mamba, the design objective is to maintain low computational overhead while preserving strong forecasting performance in multivariate TSF (Wang et al., 2024). The paper motivates the model by distinguishing two kinds of structure in forecasting: temporal dependency (TD) within each variate and inter-variate correlation (VC) across variates. Its central architectural claim is that Mamba is most useful for the latter.

The model is deliberately minimal. Its pipeline is:

1. transpose the input from $(B,L,V)$ to $(B,V,L)$, so each variate’s history becomes one token-like vector;
2. apply a linear layer independently to each variate to obtain

$\bm{U}^{tok}\in\mathbb{R}^{B\times V\times D};$

1. pass these variate tokens through one or more bidirectional Mamba VC encoding layers;
2. use an FFN TD encoding layer;
3. project to horizon length $T$, then transpose back to $(B,T,V)$ (Wang et al., 2024).

The bidirectional Mamba encoder is defined by forward and backward Mamba blocks,

$$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$0

followed by fusion

$$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$1

The paper interprets this as restoring global inter-variate information, because a unidirectional scan over variates would otherwise encode only one directional dependency pattern (Wang et al., 2024).

The Mamba block itself is described in standard selective-SSM terms. The paper gives the continuous-time SSM

$$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$2

and the discretized form

$$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$3

with

$$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$4

It further states that Mamba uses input-dependent $$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$5, $$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$6, and $$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$7, implemented through linear projections and a selective scan (Wang et al., 2024).

A distinctive feature of Simple-Mamba is its division of labor. Rather than using Mamba everywhere, it assigns VC encoding to bidirectional Mamba and TD encoding to an FFN. The paper’s ablations support this design: replacing the TD FFN with uni-Mamba, bi-Mamba, or attention degrades performance on several datasets, especially Electricity and Traffic (Wang et al., 2024). This suggests that, within this architecture, Mamba’s comparative advantage lies in efficient cross-variate fusion rather than direct temporal-sequence modeling.

3. Empirical profile of Simple-Mamba

Simple-Mamba is evaluated on 13 public datasets spanning traffic, electricity, weather, exchange, solar, and ETT benchmarks (Wang et al., 2024). The forecasting protocol uses lookback $$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$8, with horizons $$U_{in}=[u_1,u_2,\ldots,u_L]\in\mathbb{R}^{L\times V}$$9 for Traffic, ETT, Electricity, Exchange, Weather, and Solar-Energy, and $$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$0 for PEMS datasets (Wang et al., 2024). Metrics are MSE and MAE.

Its strongest results occur on high-variate or strongly periodic datasets. On Traffic, it is best on all four horizons, with average 0.414 / 0.276, compared with 0.428 / 0.282 for iTransformer and 0.481 / 0.304 for PatchTST (Wang et al., 2024). On PEMS04, PEMS07, PEMS08, Electricity, and Weather, it also achieves the best average results reported in the paper (Wang et al., 2024). On Solar-Energy, it remains competitive but is slightly behind iTransformer on average, and on ETT and Exchange its advantage narrows or disappears (Wang et al., 2024).

The paper explicitly connects this pattern to data structure. S-Mamba is said to excel when there are many variates and meaningful cross-variate correlations, especially periodic ones; when there are few variates or weak VC structure, the bidirectional Mamba encoder contributes less and may introduce noise (Wang et al., 2024).

Efficiency is part of the model’s argument. The paper reports experiments on a single NVIDIA RTX3090 GPU with batch size 16, and states that S-Mamba uses shorter training time and lower GPU memory usage than Transformer-based models on datasets such as PEMS07, Electricity, Traffic, and ETTm1 (Wang et al., 2024). It does not provide an exact closed-form complexity formula for S-Mamba itself, but repeatedly characterizes Mamba-based processing as having near-linear complexity, in contrast to Transformer attention’s $$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$1 cost (Wang et al., 2024).

The paper also explores broader questions about Mamba in forecasting. It reports that S-Mamba is largely unaffected by variate order after training, that replacing Transformer encoders with uni-Mamba in several models can reduce memory and training time while slightly improving overall performance, and that increasing lookback length helps S-Mamba and iTransformer but does not fully solve the long-lookback problem (Wang et al., 2024). These findings position Simple-Mamba not merely as a new baseline but as an argument for where Mamba is effective in TSF.

4. Sparse-Mamba as a control-structured S-Mamba

A second exact use of S-Mamba is Sparse-Mamba, which is conceptually very different from Simple-Mamba (Hamdan et al., 2024). Rather than addressing forecasting, it reparameterizes Mamba and Mamba2 using classical control-theoretic structure. The paper introduces:

• SC-Mamba: Sparse Controllable Mamba,
• SO-Mamba: Sparse Observable Mamba,
• ST-Mamba2: Sparse Stable Mamba2 (Hamdan et al., 2024).

Its starting point is the continuous-time state-space model

$$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$2

and the discrete form

$$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$3

The paper argues that vanilla Mamba does not explicitly enforce controllability or observability, and that Mamba2 does not always guarantee stability in practice (Hamdan et al., 2024).

For SC-Mamba, the state matrix $$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$4 is placed in controllable canonical form,

$$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$5

with

$$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$6

For SO-Mamba, $$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$7 is placed in observable canonical form, with the usual dual arrangement of $$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$8, $$U_{out}=[u_{L+1},u_{L+2},\ldots,u_{L+T}]\in\mathbb{R}^{T\times V}.$$9, and $A$0 (Hamdan et al., 2024).

The paper’s notion of sparsity is structural. In these canonical forms, the $A$1 matrix $A$2 has only $A$3 free parameters, because most entries are fixed zeros or ones (Hamdan et al., 2024). For ST-Mamba2, stability is enforced by clamping diagonal entries of $A$4 so that

$A$5

Since the matrix is diagonal in the paper’s Mamba2 setting, this guarantees negative eigenvalues (Hamdan et al., 2024).

Empirically, the paper evaluates on CodeParrot, OpenWebText, ArXiv, and Cosmopedia, reporting improved perplexity and modest training-time reductions. For example, on CodeParrot 1M, perplexity is 10.46 for Mamba, 10.05 for SO-Mamba, and 9.89 for SC-Mamba; parameter counts are 64,475,648 for Mamba, 64,352,904 for SO-Mamba, and 64,344,840 for SC-Mamba (Hamdan et al., 2024). The paper presents this as evidence that explicit control-theoretic structure can yield more efficient and better-behaved SSMs.

This suggests a broader interpretation of “S-Mamba”: not just Simple-Mamba, but also Sparse-Mamba, where the “S” denotes a sparse, control-structured state matrix. The two uses are unrelated in application domain but share the Mamba/SSM substrate.

5. Related domain-specific forms and frequent confusions

Several papers complicate the term’s interpretation by introducing closely related names. In hyperspectral image classification, SS-Mamba stands for “Spectral-Spatial Mamba” (Huang et al., 2024). It operates on an HSI cube $A$6, generates spatial tokens and spectral tokens, and processes them with stacked spectral-spatial Mamba blocks, each containing two basic Mamba blocks and a spectral-spatial feature enhancement module (Huang et al., 2024). The paper reports competitive results on Indian Pines, Pavia University, and Houston 2013, with the model explicitly positioned as a first attempt to apply Mamba to HSI classification (Huang et al., 2024).

A related but distinct model is S$A$7Mamba, a Spatial-spectral State Space Model for Hyperspectral Image Classification (Wang et al., 2024). It applies one selective SSM over spatial scan routes through a Patch Cross Scanning module and another over spectral bands through a Bi-directional Spectral Scanning module, then fuses them with a Spatial-spectral Mixture Gate (Wang et al., 2024). On Indian Pines, Pavia University, and Houston 2013, it reports OA = 97.92, 97.81, and 93.36, respectively (Wang et al., 2024).

In speech technology, Speech-Mamba is explicitly described as a reasonable instance of “S-Mamba” in the sense of a speech-oriented selective state space model (Gao et al., 2024). It is a joint CTC/attention encoder-decoder model for automatic speech recognition, with one Mamba encoder and one Mamba decoder, evaluated on LibriSpeech and specially constructed long-form sets up to roughly 100 seconds (Gao et al., 2024). On 100-hour LibriSpeech training, it reports strong long-context gains: for example, on test-clean-L, Transformer gives 48.66 WER while Speech-Mamba gives 7.71 (Gao et al., 2024).

The survey “Vision Mamba: A Comprehensive Survey and Taxonomy” also highlights a naming ambiguity: it does not explicitly name a model exactly called “S-Mamba,” but it lists SS-Mamba, S2Mamba, Samba, and other related variants in the visual domain (Liu et al., 2024). Its synthesis identifies SS-Mamba as the strongest exact-name confusion with “S-Mamba” in that literature (Liu et al., 2024).

Taken together, these works show that “S-Mamba” is often a shorthand or mistaken reference to one of several specialized Mamba architectures, especially in speech, hyperspectral imaging, or sparse/control-structured sequence modeling.

6. Conceptual significance and limitations

Across its multiple uses, S-Mamba consistently denotes an attempt to specialize Mamba’s selective state-space modeling to a concrete modeling bottleneck. In Simple-Mamba, that bottleneck is efficient encoding of inter-variate correlations in multivariate forecasting (Wang et al., 2024). In Sparse-Mamba, it is the lack of explicit controllability, observability, and stability guarantees in the SSM parameterization (Hamdan et al., 2024). In related forms such as SS-Mamba, S$A$8Mamba, and Speech-Mamba, the bottleneck is domain structure: spatial-spectral coupling in HSI or long-context dependency propagation in speech (Huang et al., 2024, Wang et al., 2024, Gao et al., 2024).

A recurring theme is that these models do not usually replace all other mechanisms with Mamba. Simple-Mamba keeps an FFN for temporal dependency (Wang et al., 2024). Speech-Mamba retains Transformer-style attention around Mamba blocks rather than being a pure Mamba architecture (Gao et al., 2024). SS-Mamba and S$A$9Mamba use dual-branch designs and explicit fusion modules rather than a monolithic scan over all dimensions (Huang et al., 2024, Wang et al., 2024). This suggests a broad design principle: Mamba is often treated as a strong sequence-mixing primitive, but many successful “S-Mamba” variants remain hybrid in function.

The limitations are equally domain-specific. Simple-Mamba’s advantage weakens on datasets with few variates and weak VC structure, such as parts of ETT and Exchange (Wang et al., 2024). Sparse-Mamba provides control-theoretic structure but leaves many implementation details underspecified and offers only preliminary experimental evidence (Hamdan et al., 2024). Speech-Mamba reports dramatic long-context gains but provides no direct runtime or memory benchmarks and is explicitly non-streaming (Gao et al., 2024). SS-Mamba and S$^2$0Mamba argue linear-complexity advantages over Transformers, yet neither paper provides comprehensive runtime tables in the supplied material (Huang et al., 2024, Wang et al., 2024).

This suggests that “S-Mamba” is best understood not as a finished, singular architecture, but as a recurring research pattern: domain-specific, task-structured reinterpretations of selective state-space modeling.

7. Legacy and subsequent developments

Simple-Mamba has already been treated as a baseline and point of departure in later TSF work. ms-Mamba, for example, is presented as a multi-scale extension of S-Mamba’s single-scale design, using multiple Mamba branches with different sampling rates $^2$1 processed in parallel (Karadag et al., 10 Apr 2025). The paper explicitly distinguishes:

• Vanilla Mamba: one Mamba stream, one effective sampling rate,
• S-Mamba: bidirectional Mamba for TSF, still essentially single-scale,
• ms-Mamba: bidirectional multi-branch Mamba with several $^2$2 values processed in parallel (Karadag et al., 10 Apr 2025).

This later positioning is important because it turns S-Mamba into a recognized reference point within forecasting research. It also clarifies how the term has been normalized in at least one subfield: in TSF, “S-Mamba” most directly means Simple-Mamba (Wang et al., 2024, Karadag et al., 10 Apr 2025).

A broader implication is that later Mamba-family research often addresses one of the same tensions visible across S-Mamba variants: the tradeoff between expressivity, efficiency, and domain alignment. That trend appears in multi-scale forecasting models such as ms-Mamba (Karadag et al., 10 Apr 2025), in control-structured models such as Sparse-Mamba (Hamdan et al., 2024), and in more foundational Mamba-line developments such as Mamba-3, which revisits recurrence expressivity, state tracking, and inference efficiency from an SSM-first perspective (Lahoti et al., 16 Mar 2026). Although Mamba-3 is not called S-Mamba, it is highly relevant when “S-Mamba” is used in the broad sense of a state-space Mamba architecture (Lahoti et al., 16 Mar 2026).

In that sense, S-Mamba occupies two roles in the literature. First, it names specific architectures, most notably Simple-Mamba and Sparse-Mamba. Second, it functions as part of a wider naming ecology in which Mamba variants are specialized by domain, inductive bias, or state-space reformulation. The enduring significance of the term lies less in a single canonical model than in what these works collectively demonstrate: that selective state-space models are most effective when their sequence dynamics are matched carefully to the structure of the target problem.