Multi-Model State Space Block
- Multi-Model State Space Block is a framework that combines multiple state-space hypotheses via fusion to yield unified estimates.
- It integrates classical IMM methods and neural architectures, using parallel filters and diverse branches to enhance performance.
- This approach underpins efficient long-range dynamic modeling across domains, from control systems to vision and language applications.
Searching arXiv for the cited papers to ground the article. “Multi-Model State Space Block” does not denote a single canonical construct across the recent arXiv literature. Instead, the phrase is used for several technically distinct patterns that share a state-space core while differing in how multiple models, modes, branches, or latent-state pathways are combined. In classical estimation, it refers most directly to a bank of parallel state-space filters with probabilistic interaction and fusion, as in the Interacting Multiple Model (IMM) method (Dingler, 2022). In neural architectures, it often denotes a state-space block that is split into multiple processing paths—such as local and global branches, multi-scale branches, or bidirectional scan orders—or a state-space component embedded inside a larger ensemble or modular system. A representative example is LE-Mamba for image fusion, where the local-enhanced vision Mamba block (LEVM) and state sharing jointly adapt vision Mamba to local texture recovery, multi-scale spatial modeling, and spectral consistency (Cao et al., 2024). Other usages include blockwise language-model hybrids, multi-resolution dense-prediction backbones, multi-modal registration and anomaly-detection systems, Kalman-filter blocks inside macro-financial ensembles, and modular control-oriented state-space building blocks (Fathi et al., 2023, Zhang et al., 2024, Wang et al., 2024, He et al., 2024, Santos, 17 Jun 2026, Herbst, 2019).
1. Conceptual scope and defining properties
In the strictest systems-theoretic sense, a multi-model state-space block is a mechanism that maintains more than one state-space hypothesis and produces a single estimate by interaction or fusion. The IMM paper states this explicitly: the block is a bank of filters plus a probabilistic fusion layer, not one filter alone (Dingler, 2022). In this usage, “multi-model” means that several candidate dynamical models coexist, compete, and are combined.
In neural-state-space papers, the term is broader. LE-Mamba uses a customized vision-Mamba block with a local VMamba branch, a global VMamba branch, and explicit state reuse across adjacent and skip-connected flows (Cao et al., 2024). Block-State Transformer combines an SSM sublayer with a Block Transformer sublayer (Fathi et al., 2023). HRVMamba uses a multi-scale, multi-branch, high-resolution state-space vision block rather than multiple independent models (Zhang et al., 2024). MambaAD describes a Locality-Enhanced State Space module whose Hybrid State Space core is combined with multi-kernel convolutions and hybrid scanning (He et al., 2024). MV-GMN uses a Bidirectional State Space Block with four scanning strategies and a GCN module to aggregate multi-view, multi-modal, and multi-temporal information (Lin et al., 23 Jan 2025).
This variation suggests that the phrase has become an umbrella label for architectures in which state-space dynamics are no longer used as a single serial recurrence, but as one component in a structured composition. The composition may be probabilistic, branchwise, scan-based, multi-scale, or ensemble-based, depending on the domain.
2. Classical interpretation: the Interacting Multiple Model block
The clearest literal realization of a multi-model state-space block is the IMM method. The motivation is that many systems do not obey one fixed dynamic model for all time: they may switch between operating modes, behave differently under different maneuvers, or change stochastically from the observer’s viewpoint. A single state-space model can then become inaccurate or even diverge, while the Multiple Model approach represents different possible behaviors with multiple candidate state-space models, estimates each in parallel, and decides probabilistically which model currently fits best (Dingler, 2022).
IMM is a second-generation MM method in which switching is modeled as a homogeneous Markov chain. For each model , the block maintains a model-conditioned estimate , covariance , and model probability . At each time step, it performs model probability prediction, interaction or mixing, model-conditioned Kalman filtering, likelihood-based probability update, and fusion into one combined estimate. The prior model probability is
and the mixing probability is
These weights define mixed initial conditions for each filter:
Each model then runs its own Kalman filter. Measurement likelihoods are Gaussian,
and Bayes’ rule updates posterior mode probabilities: The final fused estimate is
0
with a corresponding mixture covariance. IMM uses only 1 filters rather than the infeasible exact 2 alternatives over 3 time steps, and this is the paper’s principal computational argument for why it is a practical but suboptimal approximation (Dingler, 2022).
The IMM formulation is the reference point against which looser neural uses of “multi-model state-space block” can be compared. In IMM, multiplicity is literal: multiple dynamic models are run in parallel and fused into one state-space estimate.
3. Fusion-oriented neural realization: LEVM and state sharing in LE-Mamba
In image fusion, the relevant multi-model/state-space block is the combination of LEVM and state sharing inside LE-Mamba. The paper argues that directly transplanting Mamba from language to vision is inadequate for fusion because the problem is not only long-range sequence dependency: it also requires strong local texture recovery, multi-scale spatial modeling, and preservation of spectral consistency across modalities (Cao et al., 2024).
The paper starts from the standard continuous and discretized SSM: 4
5
For efficient computation, Mamba is written in convolutional form,
6
with kernel coefficients derived from 7, 8, and 9. Two limitations are emphasized. First, standard Mamba has limited state capacity, so information stored in the state can be lost as sequence length grows. Second, it mainly captures spatial dependencies, whereas image fusion also needs explicit handling of spectral information. The paper therefore does not simply increase the state dimension 0: its ablation shows that increasing 1 from 2 to 3 yields only limited gains while increasing memory and FLOPs (Cao et al., 2024).
LEVM injects local inductive bias into VMamba while retaining global context modeling. It contains a local VMamba block and a global VMamba block. The local branch partitions the image into windows,
4
processes them with VMamba,
5
then merges them back,
6
followed by a global VMamba block,
7
Two hidden states, 8 and 9, are maintained. The local branch learns detailed spatial structures, and the global branch reintegrates them to improve the representation of the whole image (Cao et al., 2024).
The second mechanism is state sharing. Hidden states are reused across adjacent layers and across encoder-decoder skip-connected flows. The key spatial-spectral learning update is
0
1
where 2 is learnable. The paper interprets 3 as a basis for spectral representation and 4 as carrying rich spatial information. Their interaction is added back residually to the input. Algorithmically, the block applies CrossScan, grouped linear projections, an 5-based S2L interaction, parameterizes 6, runs SS2D, and then applies CrossMerge and an FFN with residual connection (Cao et al., 2024).
The full LE-Mamba network is a multi-scale encoder-decoder architecture. The encoder uses
7
and the decoder uses
8
The final output is mapped back to pixel space by a linear layer, with LRMS added following a high-frequency learning strategy. Training uses
9
with 0 (Cao et al., 2024).
The ablation study isolates the block design on WV3:
| Variant | WV3 metrics | Interpretation |
|---|---|---|
| NAFBlock baseline | SAM 3.52; ERGAS 2.66; Q2n 0.902; SCC 0.977 | multi-scale baseline |
| Vanilla VMamba | SAM 2.93; ERGAS 2.15; Q2n 0.910; SCC 0.987 | global SSM alone |
| LEVM | SAM 2.86; ERGAS 2.17; Q2n 0.914; SCC 0.986 | local + global branch |
| LEVM + adjacent flow | SAM 2.80; ERGAS 2.08; Q2n 0.918; SCC 0.987 | state sharing across layers |
| LEVM + skip-connected flow | SAM 2.76; ERGAS 2.02; Q2n 0.921; SCC 0.988 | best ablation result |
A separate control shows that the gain is not due to a generic residual path: vanilla LEVM gives SAM 1, a residual with 2 gives SAM 3, and the proposed 4 gives SAM 5 (Cao et al., 2024). Within this paper, the multi-model/state-space block is therefore a fusion-specific redesign of vision Mamba, not an ensemble of separate models.
4. Parallelizable sequence and MIMO state-space block designs
In language modeling, the Block-State Transformer is a hybrid layer that combines an SSM sublayer for long-range contextualization with a Block Transformer sublayer for short-term sequence representation. The SSM is written as
6
and unrolled into a convolution 7, computed in parallel via FFT. The sequence is partitioned into blocks of length 8, and the block transformer cross-attends to SSM-derived context states. The paper studies three fully parallelizable variants—SH, MH, and MF—and gives the layer runtime as
9
It reports more than tenfold increase in speed at the layer level compared to the Block-Recurrent Transformer when model parallelization is employed, and shows that structured S4-based variants generalize better to longer sequences such as 0 tokens (Fathi et al., 2023).
A different line of work addresses multiplicity at the level of channel structure rather than branch composition. The efficient SSM paper proposes a MIMO state space layer, eSSM, instead of stacking many SISO branches. The continuous model is
1
with 2, 3, and 4. After discretization and unrolling, the system becomes a convolution. The principal efficiency strategy is diagonalization of 5, which decouples the dynamics and reduces the time complexity of constructing 6 from 7 to 8, followed by FFT-based per-channel convolution. The paper states that the convolution cost drops from
9
to
0
It reports that the parameters of eSSM are 1 of LSTM and 2 of Mamba, and that training speed is 3 times faster than LSTM and 4 times faster than Mamba (Liang et al., 2024).
These two papers illustrate two non-IMM meanings of a multi-model state-space block. In BST, multiplicity arises from a parallel SSM-plus-attention composition. In eSSM, it arises from a direct multi-input multi-output state-space formulation with diagonalized latent dynamics.
5. Multi-scale, multi-branch, and multi-modality visual state-space blocks
Several vision papers use the phrase in a looser sense to denote a state-space block that has multiple branches, scan orders, or modalities rather than multiple independent models. VMambaMorph, for example, is explicitly described as not a “multi-model” block in that strict sense. It is a multi-modality deformable image registration framework whose core block is a 3D redesign of the Visual State Space block with a cross-scan module. The method uses a U-shaped hybrid CNN + VMamba encoder-decoder, predicts a deformation field 5, and adds recursive refinement
6
with the best configuration reported as 7. On the SR-Reg MR-CT dataset, the best table entry reports Dice 8, HD9 0, and 1 2 (Wang et al., 2024).
HRVMamba introduces the Dynamic Visual State Space block for dense prediction. Its DSS2D component replaces the depthwise convolution in SS2D with 3 DCNv4,
4
while its MultiDW component splits channels into 5 groups and applies depthwise kernels of size 6, 7, 8, and 9. The block is embedded in an HRNet-style four-stage high-resolution parallel backbone. The paper states that replacing SS2D with DSS2D improves AP by 0, and that multi-scale kernels rather than a single 1 convolution are responsible for the gains (Zhang et al., 2024).
MambaAD uses a Locality-Enhanced State Space module whose global branch is a cascade of Hybrid State Space blocks,
2
and whose local branch uses two parallel depth-wise convolutions with kernel sizes 3 and 4,
5
The outputs are fused by
6
The paper further proposes Hybrid Scanning with five methods—Sweep, Scan, Z-order, Zigzag, and Hilbert—and eight directions, arguing that Hilbert scanning is especially effective for preserving locality while enabling long-range traversal (He et al., 2024).
MV-GMN extends the pattern to multi-view action recognition. Its Bidirectional State Space Block applies four scan strategies—view-prioritized forward and backward, and time-prioritized forward and backward—to fused RGB-skeleton features, then combines the result with a GCN whose graph edges are constructed by rule-based and KNN-based methods. The block processes scan sequences by a local convolution, a projection, an SSM stage, and a residual-style combination,
7
On NTU RGB+D 120, the paper reports 8 cross-subject and 9 cross-view accuracy for MV-GMN (Lin et al., 23 Jan 2025).
Across these papers, “multi-model” usually does not mean an IMM-style bank of separate dynamical models. It more often means that state-space computation is embedded in a composite visual block that is multi-scale, multi-branch, multi-directional, or multi-modal.
6. Ensemble and modular system interpretations
A further usage appears in block ensembles. The Brazil neutral-rate paper defines a five-block architecture consisting of simple, statistical, market, state-space market, and semi-structural blocks. The state-space portion is a pair of Kalman-filter-based components: a yield-curve state-space model and a semi-structural IS-Phillips state-space model. Block medians are combined by
0
In the reported run, the effective weights are Simple 1, Statistical 2, Market 3, State-space market 4, and Semi-structural 5, because the semi-structural Kalman implementation falls back to a local-level trend rather than a valid structural estimate. The May 2026 estimate places the final operational neutral-rate proxy at 6 p.a., with a P25-P75 block range of 7, and the yield-curve state-space output at 8 (Santos, 17 Jun 2026). Here, the state-space block is one component inside a broader auditable ensemble, and the methodological emphasis is on conservative inclusion rather than unconditional model averaging.
An older control-oriented systems paper shows another block-based interpretation. It proposes a building-block approach to state-space modeling of DC-DC converter systems in which passive components, controlled converters, and controllers are modeled as small state-space subsystems and connected by standardized operations. The central two-port converter description is
9
00
Controllers can be attached through an open-loop or closed-loop construction, and two-port subsystems can be series-connected by eliminating internal port variables. Because the assembled model remains a two-port state-space system, control-to-output transfer functions, output impedance, input admittance, forward voltage gain, and reverse current gain can be extracted at any modeling stage (Herbst, 2019).
These examples broaden the term beyond deep learning. A multi-model state-space block may also be an ensemble component with explicit validity checks, or a reusable subsystem in modular system assembly.
7. Recurrent misconceptions and comparative perspective
A recurring misconception is that every state-space block with more than one branch is “multi-model” in the strict IMM sense. The surveyed papers do not support that equivalence. VMambaMorph explicitly states that its contribution is not a “multi-model” block in the sense of using multiple separate models, but a 3D redesign of the VMamba/VSS block for multi-modality registration (Wang et al., 2024). HRVMamba likewise states that it does not introduce a multi-model block in the sense of combining multiple pretrained models, but a multi-scale, multi-branch, high-resolution state-space vision block (Zhang et al., 2024). MambaAD’s LSS and HSS blocks, and MV-GMN’s Bidirectional State Space Block, similarly use multiplicity through scan paths, branches, kernels, or modalities rather than through a bank of separate state estimators (He et al., 2024, Lin et al., 23 Jan 2025).
A second misconception is that enlarging latent state capacity alone solves the adaptation problem. LE-Mamba directly tests this claim and reports only limited SAM improvement from 01 to 02 while increasing memory and FLOPs, whereas the architectural modifications of LEVM and state sharing yield larger gains (Cao et al., 2024). A related point appears in HRVMamba and MambaAD: both papers argue that standard scan-based SSMs require additional local inductive bias for dense or anomaly-sensitive vision tasks, achieved respectively through MultiDW plus DCNv4 and through multi-kernel local branches plus hybrid scanning (Zhang et al., 2024, He et al., 2024).
A third misconception is that state-space outputs should always be interpreted as structurally valid latent variables. The Brazil neutral-rate paper rejects this: the semi-structural IS-Phillips Kalman block receives zero weight because it falls back to a local-level trend, and the resulting 03 estimate is explicitly not labeled a long-run structural neutral rate (Santos, 17 Jun 2026). In this case, the “multi-model state-space block” is valuable precisely because invalid structural content is excluded rather than forced into the final estimate.
Taken together, these works indicate that the expression “Multi-Model State Space Block” is best treated as a family resemblance term. Its strictest and most literal meaning is the IMM-style bank of interacting filters. Its most common modern neural meaning is a composite state-space module in which multiple branches, scan orders, scales, or latent-state flows are organized around an SSM core. Its broader systems interpretation includes block ensembles and modular interconnection frameworks. The unifying feature is not a single architecture, but the use of state-space dynamics as the organizing substrate for structured composition, selective fusion, and efficient long-range or latent-state modeling.