---
title: Adaptive Scanning State Space Model
url: https://www.emergentmind.com/topics/adaptive-scanning-state-space-model
type: topic
---

# Adaptive Scanning State Space Model

An “adaptive scanning state space model” (*Editor’s term*) denotes a class of state space model designs in which performance depends critically on how non-sequential data are serialized, traversed, or read, and in which that scanning step is modified to better preserve locality, semantics, scale, temporal correspondence, or multimodal structure. In recent Mamba-derived literature, this modification appears in several distinct forms: fixed but geometry-aware scans such as Hilbert/fractal traversals, layerwise searched local scans, content-aware spatial resampling, graph-derived spatiotemporal orderings, query-conditioned state updates, and adaptive fusion/readout built on top of a fixed scan [2405.14480][2403.09338][2502.12627][2506.11768].

## 1. Sequence modeling foundations and the scanning problem

Recent vision and event-based SSM work is built on the standard continuous-time state space formulation
$$
h'(t)=\mathbf{A}h(t)+\mathbf{B}x(t),\qquad y(t)=\mathbf{C}h(t),
$$
with zero-order-hold discretization
$$
\overline{\mathbf{A}}=\exp(\Delta\mathbf{A}),\qquad
\overline{\mathbf{B}}=(\Delta \mathbf{A})^{-1}\big(\exp(\Delta \mathbf{A})-\mathbf{I}\big)\Delta \mathbf{B},
$$
and discrete recurrence
$$
h_t=\overline{\mathbf{A}}h_{t-1}+\overline{\mathbf{B}}x_t,\qquad y_t=\mathbf{C}h_t.
$$
Across the Mamba lineage, the decisive modification is that \(\mathbf{B}\), \(\mathbf{C}\), and \(\Delta\) become input-dependent, turning the model from an LTI system into a selective or time-varying SSM [2405.14480][2403.09338].

For images, videos, burst stacks, hyperspectral cubes, and event streams, the central difficulty is that the input is not natively a 1D causal sequence. Visual SSMs therefore require a 2D-to-1D or spatiotemporal-to-1D serialization step. The papers consistently argue that this step is not an incidental preprocessing choice. In a selective scan, sequence order determines which spatial or temporal relations are nearby in recurrence time, which dependencies are easy to propagate, and which local structures are artificially fragmented by serialization [2405.14480][2403.09338][2501.15455].

This leads to a common diagnosis across otherwise different application domains. Raster-like, row-wise, column-wise, zigzag, or fixed directional scans can preserve some adjacency while breaking other adjacency relations; fixed local windows improve locality but introduce rigid partitions; and arbitrary flattening can mix semantically unrelated tokens while separating tokens that belong to the same object, change region, or motion trajectory. Adaptive-scanning research therefore treats scan design as a first-order modeling variable rather than an implementation detail [2502.12627][2501.15455][2506.11768].

## 2. Main forms of scan adaptation

A practical taxonomy suggested by the literature separates scan adaptation into several distinct mechanisms rather than a single design pattern.

| Form | Core mechanism | Representative papers |
|---|---|---|
| Structural or geometric adaptation | Fixed recursive or multi-rate scans that better preserve locality or scale consistency | FractalMamba [2405.14480], AtrousMamba [2507.16172], MambaADv2 [2606.23126] |
| Layer-adaptive scan selection | Search over scan candidates per layer or stage | LocalMamba [2403.09338] |
| Content-aware scan construction | Input-dependent offsets, graph ordering, or selected-frame paths define the scan | DAMamba [2502.12627], MambaVSR [2506.11768], PAST-SSM [2409.16953], CD-Lamba [2501.15455] |
| Adaptive read/fusion on top of fixed scan | Keep write scan fixed, but make readout or branch fusion adaptive | Deformba [2605.21308], QMambaBSR [2408.08665], LEAF-Mamba [2509.18683] |
| Physical-scan restoration with adaptive SSM processing | “Scanning” refers primarily to the acquisition process, while adaptation occurs inside restoration | RCL-Mamba [2606.31353] |
| Scanning-free alternatives | Remove scan serialization entirely and replace it with spectral processing | HAMSA [2604.14724] |

The literature is explicit that these forms are not interchangeable. FractalMamba is adaptive to resolution and hierarchy through recursive Hilbert construction, but not adaptive to image content [2405.14480]. LocalMamba adapts scan choice per layer through differentiable search, but the deployed model uses fixed selected directions at inference [2403.09338]. DAMamba and MambaVSR are closer to fully content-aware scan construction because the scan depends on the current input feature map or semantic graph [2502.12627][2506.11768]. Deformba, by contrast, states that the write scan remains fixed and that adaptivity is introduced during the read stage [2605.21308].

This distinction is central to the field. “Adaptive scanning” may refer to structural scale adaptivity, architecture-level scan selection, content-conditioned traversal, or adaptive state fusion that weakens dependence on scan order without literally changing the write pass.

## 3. Representative mechanisms

The most direct structural approach is the recursive fractal scan. FractalMamba replaces linear image serialization with a Hilbert curve–based scan. The recursive definition
$$
\texttt{Hilbert}(x_0,y_0,\vec{x},\vec{y},depth)
$$
subdivides a square into four quadrants, swaps and halves \(\vec{x}\) and \(\vec{y}\), and emits the center point at depth zero. This yields a 1D ordering that preserves locality and self-similarity across resolutions, and the model uses four fractal curves in four directions inside a FractalMamba block [2405.14480].

LocalMamba introduces a different mechanism: the scan candidates themselves form an architecture search space. The candidate set contains horizontal, flipped horizontal, vertical, flipped vertical, local-\(2\), flipped local-\(2\), local-\(7\), and flipped local-\(7\), and the layer output during search is a softmax mixture
$$
\bm{y}^{(l)}=\sum_{s\in\mathcal{S}}
\frac{\exp(\alpha_s^{(l)})}{\sum_{s'\in\mathcal{S}}\exp(\alpha_{s'}^{(l)})}
\mathrm{SSM}_s(\bm{x}^{(l)}).
$$
After search, the top 4 directions are retained per layer [2403.09338].

DAMamba moves to genuinely input-conditioned spatial resampling. Its Dynamic Adaptive Scan predicts offsets
$$
\Delta p=\mathrm{OPN}([x_1,x_2,\ldots,x_N]),
$$
updates reference coordinates
$$
p'[h,w,:]=p[h,w,:]+\Delta p[h,w,:],
$$
and samples adaptively by bilinear interpolation,
$$
X'[h,w]=\zeta(p'[h,w],X).
$$
The sequence is then flattened in original row-major order, but each slot now carries content sampled from a data-adaptively shifted 2D location [2502.12627].

MambaVSR constructs a spatiotemporal scan from semantic graphs. Its Shared Compass Construction builds an intra-frame semantic graph, forms a Laplacian, extracts the Fiedler vector, and sorts nodes to obtain an adaptive spatial order \(\mathcal{O}\). Content-Aware Sequentialization then interleaves aligned multi-frame features along that order, producing a sequence in which semantically corresponding content across frames becomes nearby in recurrence space [2506.11768].

CD-Lamba adapts scanning at the level of local change regions. A score map over \(4\times4\) coarse windows is computed from a bi-temporal difference cue, top-\(k\) windows are selected by Gumbel Softmax, connected windows are merged, and the final sequence is assembled as
$$
S=S_0\oplus S_1\oplus\cdots\oplus S_{k'},
$$
where \(S_0\) is the non-selected background/global sequence and \(S_i\) are connected local change-region sequences. CTSS then interleaves aligned bi-temporal tokens pixel by pixel before S6 processing [2501.15455].

Deformba provides a different mechanism altogether. It keeps a single fixed write scan,
$$
\mathbf{S}_t=\alpha_t\mathbf{S}_{t-1}+\mathbf{V}_t\mathbf{K}_t^\top,
$$
materializes a 2D state map \(\mathbf{S}_{2D}\), predicts offsets \(\Delta p\), forms sampling positions \(\mathcal{P}=\mathcal{E}+\Delta p\), and performs adaptive state fusion
$$
S_Q=\sum_{g=1}^{G} w_g\cdot \phi(S_{2D},\mathcal{P}_g).
$$
The effective interaction graph is therefore
$$
G_{eff}=G_{write}\cup G_{read},
$$
so scan order no longer solely determines spatial interaction [2605.21308].

PAST-SSM shows the same idea in event streams. Path-Adaptive Event Aggregation and Scan learns a selection mask \(M\in\mathbb{R}^{K\times P}\) over \(P=Tf\) aggregated event frames, uses Gumbel-Softmax during training, sorts selected indices in temporal order, and yields a fixed-length selected sequence \(F'\in\mathbb{R}^{K\times H\times W\times 3}\) before bidirectional Mamba processing [2409.16953].

## 4. Domain-specific instantiations

In generic visual backbones, adaptive scanning is primarily a response to the mismatch between sequence models and 2D structure. FractalMamba, LocalMamba, DAMamba, MFil-Mamba, AtrousMamba, S\(^2\)Mamba, and MambaADv2 all treat scan organization as a core design variable, but they do so with different task priors: resolution generalization and Hilbert locality in FractalMamba; layerwise scan search in LocalMamba; dynamic adaptive scan regions in DAMamba; multi-filter scan diversity in MFil-Mamba; atrous-window selective scan for dense remote-sensing prediction in AtrousMamba; spatial and spectral scanning in hyperspectral classification for S\(^2\)Mamba; and semantics-adaptive progressive scanning over a feature pyramid in MambaADv2 [2405.14480][2403.09338][2502.12627][2603.20074][2507.16172][2404.18213][2606.23126].

In restoration and sequence-to-image problems, the scan often becomes explicitly cross-frame or cross-domain. QMambaBSR uses a Query State Space Model in which current-frame features drive the recurrence but base-frame features generate \(B_{\text{base}}\) and \(\Delta_{\text{base}}\), yielding
$$
h_t=(\overline{A}_{\text{base}_t})h_{t-1}+(\overline{B}_{\text{base}_t})x_{\text{cur}_t},
$$
so adaptivity lies in query-conditioned selective dynamics rather than scan-path learning [2408.08665]. MambaVSR instead makes the sequence itself content-aware by graph-derived spatial ordering and temporal interleaving across frames [2506.11768].

In multimodal or cross-stream settings, adaptive scanning often shifts toward adaptive fusion. LEAF-Mamba’s LE-SSM redesigns SS2D as a fixed multi-scale windowed scan, but its Adaptive Fusion Module is the more adaptive component: covariance-based second-order pooling produces similarity maps, distance maps drive complementary cross-modality interaction, and similarity maps drive reliable enhancement by modulating \(\mathbf{C}\)-related terms inside SSM-style recurrences [2509.18683]. Deformba extends this logic further by making the read stage deformable and queryable, including cross-attention-like BEV retrieval from an SSM state map [2605.21308].

A distinct but important nearby usage appears in measurement-oriented imaging. RCL-Mamba is about rotational scanning computed laminography, where “scanning” primarily denotes the physical acquisition process. Its adaptivity is architectural: a Mamba-CNN dual-branch module uses input-dependent \(B\), \(C\), and \(\Delta\) in the Mamba branch and a dynamic gated fusion
$$
M=\sigma(\mathrm{Linear}([F_{\mathrm{global}},F_{\mathrm{local}}])),
$$
followed by
$$
F_{\mathrm{out}}=\mathrm{Norm}(M\odot F_{\mathrm{local}}+(1-M)\odot F_{\mathrm{global}}),
$$
to balance blur correction and local detail recovery in a dual-domain restoration chain [2606.31353].

## 5. Empirical behavior

The literature reports that scan design materially changes both accuracy and robustness, but the magnitude of the effect depends on the task and on what is being adapted.

For fixed-image classification, purely structural scan changes often yield modest gains at standard resolution but larger gains under scale shift. FractalMamba-T reports 82.7% top-1 on ImageNet-1K at \(224^2\), comparable to VMamba-T at 82.5, but under resolution generalization from training at \(224\times224\) to testing at \(1024\times1024\), FractalMamba reaches 69.6 while VMamba reaches 62.3 and Swin 61.9. With the curve-shifting variant, the \(1024^2\) result rises to 70.4 [2405.14480].

Layer-adaptive local scanning also yields consistent gains. LocalVim-T improves over Vim-Ti from 73.1 to 76.2 at the same 1.5G FLOPs, and LocalVMamba-T improves over VMamba-T from 82.2 to 82.7. The ablations show that local scan alone gives a +1.0 gain on Vim-T, while layerwise scan search adds a further +0.4 over the fixed multi-scan design [2403.09338].

Input-conditioned scan construction is empirically stronger. DAMamba-T reports 83.8% top-1 versus 82.6 for VMamba-T, and on ADE20K single-scale mIoU it reports 50.3 versus 48.0 for VMamba-T. Its ImageNet ablation shows that adding DAS to a sweeping-scan baseline changes 24.7M parameters and 4.66G FLOPs at 80.8% top-1 into 25.1M and 4.73G at 81.2% [2502.12627].

For dense change detection, locality-adaptive sequence construction shows especially large margins. CD-Lamba reports F1 scores of 92.51 on WHU-CD, 82.66 on SYSU-CD, 71.66 on DSIFN-CD, and 78.06 on CLCD, outperforming ChangeMamba by +2.43, +3.28, about +5.75, and +8.06 F1, respectively. The scan ablation on CLCD gives 76.46 for VMamba scan, 77.56 for LocalMamba scan, and 78.06 for CT-LASS [2501.15455].

For video restoration, content-aware spatiotemporal scanning also yields measurable gains. MambaVSR reports that it outperforms the Transformer-based VRT by 0.58 dB PSNR on REDS with 55% fewer parameters, and its scan-order ablation shows raster-based scanning at 31.68 on REDS4, Fiedler-based scanning at 31.73, and full content-aware scanning at 31.82 [2506.11768].

Event-based recognition provides another strong case because sequence length varies with duration and sampling frequency. PAST-SSM reports improvements of +3.45%, +0.38%, and +8.31% on DVS Action, SeAct, and HARDVS, respectively, and markedly smaller drops under inference-frequency mismatch than fixed temporal sampling baselines [2409.16953].

Not all scan adaptations yield large standalone gains. In MambaADv2, semantics-adaptive progressive scanning contributes +0.1 mAD and reduces FLOPs by 0.2G with +1 FPS, indicating that stage-wise scan-budget scheduling is an incremental but efficient refinement on top of the stronger gains from SSD/RoPE and spatial-frequency enhancements [2606.23126].

## 6. Conceptual boundaries and open directions

A recurring misconception is that every adaptive-scanning SSM learns a token-wise traversal policy. The literature shows a broader landscape. FractalMamba is adaptive in a geometric and multiscale sense, but the scan order is fixed once resolution and orientation are specified [2405.14480]. LocalMamba adapts per layer during differentiable search, yet the resulting inference-time scans are static [2403.09338]. Deformba explicitly states that it keeps the SSM write pass fixed and introduces adaptivity during the read stage instead [2605.21308]. QMambaBSR is adaptive in state-update dynamics through query-conditioned \(B\) and \(\Delta\), not in scan-path learning [2408.08665]. LEAF-Mamba is closer to adaptive SSM fusion than adaptive scanning proper [2509.18683].

A second misconception is that “scanning” always means token serialization. In RCL-Mamba, scanning primarily refers to physical rotational scanning in computed laminography, and the adaptive element lies in restoration architecture rather than acquisition-policy learning [2606.31353]. Conversely, HAMSA argues that scan-based visual SSMs inherit an unnecessary serialization bottleneck and removes scanning entirely by replacing it with spectral adaptive gating, positioning itself as a scanning-free alternative rather than an adaptive-scanning method [2604.14724].

The open problems suggested by these papers cluster around three directions. One is stronger input-conditioned serialization: several works imply that content-adaptive routing remains underexplored beyond offsets, graph ordering, or selected-frame paths [2502.12627][2506.11768]. A second is better alignment between scan geometry and domain physics, as in RCL-Mamba’s explicit suggestion that polar-coordinate transformations may better linearize arc-shaped blur trajectories [2606.31353]. A third is reducing the search or implementation burden of adaptive scan strategies, since LocalMamba notes the memory cost of supernet-style scan search and MambaADv2 treats adaptive scan budgeting as a lightweight schedule rather than a fine-grained controller [2403.09338][2606.23126].

Taken together, these works suggest that adaptive scanning is not a single algorithmic primitive but a design space around the 2D-to-1D interface of selective SSMs. In some models, the scan is made more locality-preserving; in others, more semantically aligned, more temporally corresponding, more query-conditioned, or less dominant because adaptive readout compensates for a fixed write path. The common thesis is that once SSMs leave native 1D domains, the scan becomes part of the model itself.

Source: https://www.emergentmind.com/topics/adaptive-scanning-state-space-model