Local Emphatic State Space Module

• LE-SSM is a state-space-based module that integrates multi-scale, directional scanning to strengthen local representations in SSM backbones.
• In LEAF-Mamba, LE-SSM applies four distinct scan paths with varied window sizes to capture both fine details and broader contextual information in RGB-D data.
• Mamba3D extends LE-SSM by combining state space modeling with explicit local operators like Local Norm Pooling, improving local geometric feature extraction.

Local Emphatic State Space Module (LE-SSM) is a state-space-based mechanism for strengthening local representation inside SSM backbones. In the literature considered here, the term is used in two closely related senses. In LEAF-Mamba, LE-SSM is a concrete module for RGB-D salient object detection that captures multi-scale local dependencies through a multi-scale, multi-directional, windowed selective scan applied separately to RGB and depth features (Wu et al., 23 Sep 2025). In Mamba3D, the same idea appears as a broader design pattern in which state space modeling is combined with explicit local operators such as Local Norm Pooling (LNP), so that local geometry is emphasized before global SSM mixing (Han et al., 2024).

1. Motivation and problem setting

LE-SSM arises from a recurring limitation in vision-oriented SSM adaptations. Standard SSMs and Vision Mamba architectures commonly adapt 1D sequential modeling to visual inputs by scanning 2D images as 1D sequences, either along rows and columns or in a windowed manner. The reported failure modes are twofold: flattening 2D images into 1D sequences often disrupts adjacency and local semantic relationships, and fixed-window variants can capture local context but fail to model multi-scale dependencies. In RGB-D salient object detection, these limitations are compounded by the need for modality-specific semantics and effective use of complementary depth cues (Wu et al., 23 Sep 2025).

The same locality problem appears in other modalities. For unordered 3D point clouds, a straightforward adoption of Mamba does not achieve satisfactory performance, because vanilla SSMs are not well-equipped for unordered sets due to sequential dependencies and the lack of explicit locality modeling. Mamba3D addresses this by introducing an explicit local block and a bidirectional SSM design, and it is described as a Local Emphatic SSM view of state-space modeling for point clouds and possibly other unordered data modalities (Han et al., 2024).

A common thread across these settings is that LE-SSM is introduced not as a rejection of state-space recurrence, but as an augmentation of it. The module preserves the linear-complexity appeal of SSMs while attempting to recover local semantics that may be weakened by purely sequential scans or by arbitrary orderings of visual or geometric data (Wu et al., 23 Sep 2025).

2. Canonical LE-SSM formulation in LEAF-Mamba

In LEAF-Mamba, LE-SSM is integrated into a dual-stream encoder, with one stream for RGB and one for depth. Each stage uses standard VMamba blocks for initial feature extraction, and in the last block of each stage the LE-SSM replaces the vanilla VMamba block to enhance local, multi-scale semantics for both streams. The core mechanism is the Multi-Scale Windowed 2D Selective Scan (MSW-SS2D), in which the input feature map is traversed in four distinct scanning directions, each with a unique window size (Wu et al., 23 Sep 2025).

Scan path	Directional form	Window size
$\mathrm{H_1}$	Horizontal	1
$\mathrm{HF_2}$	Horizontal Flipped	2
$\mathrm{V_4}$	Vertical	4
$\mathrm{VF_8}$	Vertical Flipped	8

For each scan direction $$\mathrm{Scan} \in \mathcal{S} = \{ \mathrm{H_1}, \mathrm{HF_2}, \mathrm{V_4}, \mathrm{VF_8} \}$$, the feature map $X$ is unfolded into a sequence according to the scan path and window size, processed by its own SSM, reshaped back into 2D positions, and then summed with the other scan outputs. The module output is

$$Y = \sum_{\mathrm{Scan} \in \mathcal{S}} \mathrm{Reshape}(\mathrm{S6}(\mathrm{Scan}(X))).$$

Each directional sequence is handled by an S6 block resembling Mamba’s selective scan block. The selective mechanism means that the parameters $\bm{C}$, $\bm{B}$, and $\Delta$ are input-dependent, providing content-aware state transitions. The reported motivation is that local neighborhoods are processed together rather than being fragmented by a single flattened traversal, while the four scan paths aggregate distinct spatial scales and directional relations with negligible overhead (Wu et al., 23 Sep 2025).

3. Local semantics, multi-scale dependence, and modality handling

LE-SSM is applied independently to both RGB and depth streams. This independent local enhancement is intended to ensure that modality-specific local structures are emphasized before cross-modality fusion. The enhanced RGB and depth features are then fused downstream through the Adaptive Fusion Module (AFM), which is outside the LE-SSM itself but is part of the larger LEAF-Mamba design (Wu et al., 23 Sep 2025).

The reported treatment of local dependence is explicitly multi-scale. Small windows, such as size 1, capture local fine detail and edge information, whereas larger windows, such as size 8, provide contextual, mid-range dependencies that are useful for objects of varying sizes. The flipped scan directions are introduced to ensure diversity and to avoid directional bias by aggregating neighborhoods adjacent in multiple major directions. The result is described as a robust local representation for each pixel obtained by fusing four parallel SSM outputs (Wu et al., 23 Sep 2025).

This structure distinguishes LE-SSM from both classic SS2D and fixed-window scans. The relevant claim is not simply that locality matters, but that preserving local adjacency alone is insufficient if scale variation is not modeled. LE-SSM therefore couples locality preservation with scale diversification, and it does so separately for each modality before any RGB-depth interaction is imposed (Wu et al., 23 Sep 2025).

4. LE-SSM as a broader design pattern in Mamba3D

Mamba3D provides a broader interpretation of LE-SSM as a conceptual fusion of SSMs with explicit local operators. The architecture introduces the Local Norm Pooling (LNP) block to extract local geometric features from point neighborhoods and a bidirectional SSM (bi-SSM) to improve global context modeling. In this account, LE-SSM is not restricted to 2D windowed scans; it also includes designs in which local propagation and aggregation are integrated directly into the state-space backbone for unordered data (Han et al., 2024).

The LNP block operates on local patches, typically built from Farthest Point Sampling and K-Nearest Neighbor grouping. It has two stages. In K-norm, neighbor features are normalized relative to the center point and then fused with the center feature through concatenation and a learnable affine transformation. In K-pooling, the fused neighbor features are aggregated back to the center with a softmax-based weighting that is intended to be more expressive than max or average pooling while maintaining permutation invariance. The role of LNP is to inject explicit local geometry, addressing a key SSM and Transformer weakness in point cloud analysis (Han et al., 2024).

Global modeling is handled by bi-SSM. Mamba3D defines

$\mathrm{HF_2}$0

Here, L+SSM is the standard token-forward SSM, and C-SSM is a backward SSM applied over feature channels rather than token order. The stated motivation is to alleviate pseudo-order reliance caused by imposing an arbitrary sequence on an unordered point set, while also enabling global reasoning across feature channels. In this formulation, Mamba3D can be viewed as a Local Emphatic SSM because expressive, locally sensitive feature extraction is integrated directly into a state-space network (Han et al., 2024).

5. Position within the unified theory of sequence models

A later theoretical framework places attention mechanisms and state space models within a single operator-based formalism. For a sequence input $\mathrm{HF_2}$1 and output $\mathrm{HF_2}$2, the framework writes

$\mathrm{HF_2}$3

where $\mathrm{HF_2}$4 is an input-dependent effective interaction operator. Two recurring construction patterns are distinguished: the Unified Factorized Framework (Explicit), associated with attention-style mixing, and Structured Dynamics (Implicit), associated with state-space recurrences. LE-SSM is positioned within the latter family, because its interaction operators are induced by state-space dynamics rather than by a purely explicit reweighting of shared value maps (Ghodsi, 17 Dec 2025).

Three theoretical results are especially relevant. The Interaction Rank Gap states that single-head factorized models are constrained to a low-dimensional operator span and cannot represent certain structured dynamical maps. The Equivalence (Head-Count) Theorem states that representing a linear SSM whose lag operators span a $\mathrm{HF_2}$5-dimensional subspace on length-$\mathrm{HF_2}$6 sequences requires and is achievable with $\mathrm{HF_2}$7 heads in the multi-head factorized class. The Gradient Highway Result states that attention layers admit inputs with distance-independent gradient paths, whereas stable linear dynamics exhibit distance-dependent gradient attenuation (Ghodsi, 17 Dec 2025).

Within this framework, LE-SSM is described as supporting higher-rank local interaction subspaces akin to SSMs, while potentially suffering from distance-dependent gradient decay. This suggests a precise trade-off: LE-SSM-like modules inherit algebraic expressivity associated with structured dynamics, but they do not automatically inherit the optimization properties of attention’s direct input-output paths. A plausible implication, explicitly noted in the same theoretical account, is that architectures interleaving attention and SSM blocks may use SSMs for rich, structured, efficient modeling of local and mid-range interactions while inserting periodic attention blocks to refresh direct-access pathways and improve long-range credit assignment (Ghodsi, 17 Dec 2025).

6. Empirical evidence and interpretive issues

In LEAF-Mamba, the empirical role of LE-SSM is documented through ablations and scan-strategy comparisons. On NJUD, the baseline is reported as $\mathrm{HF_2}$8 and $\mathrm{HF_2}$9, while Baseline + LE-SSM reaches $\mathrm{V_4}$0 and $\mathrm{V_4}$1. On SSD, the baseline is $\mathrm{V_4}$2 and $\mathrm{V_4}$3, while Baseline + LE-SSM reaches $\mathrm{V_4}$4 and $\mathrm{V_4}$5. The reported relative improvements are +1.4% $\mathrm{V_4}$6 and -6.7% MAE on NJUD, and +2.5% $\mathrm{V_4}$7 and -11.4% MAE on SSD (Wu et al., 23 Sep 2025).

Strategy	NJUD $\mathrm{V_4}$8	SSD $\mathrm{V_4}$9
SS2D	0.917	0.851
Continuous scan	0.919	0.856
Fixed windowed scan	0.925	0.863
MSW-SS2D (LE-SSM)	0.931	0.872

These comparisons address a common misunderstanding: LE-SSM is not equivalent to any local scan or any windowed SSM. The reported results distinguish classic row/column scan, continuous scan, fixed windowed scan, and multi-scale windowed scan, with MSW-SS2D yielding the highest $\mathrm{VF_8}$0 on both large and small benchmarks. The qualitative evidence is summarized as sharper predictions, more accurate object boundaries, and better detection of objects at varying scales and in complex backgrounds. At the system level, LEAF-Mamba is reported to consistently outperform 16 state-of-the-art RGB-D SOD methods in both efficacy and efficiency, and also to achieve excellent performance on RGB-T SOD (Wu et al., 23 Sep 2025).

The related point-cloud formulation in Mamba3D supplies a second empirical perspective on local-emphatic SSM design. Mamba3D reports 92.6% overall accuracy from scratch on ScanObjectNN and 95.1% with single-modal pre-training on ModelNet40, both with linear complexity. It also reports that removing LNP leads to a 1.2–2.9% drop in accuracy, that removing bi-SSM or replacing it with attention leads to consistent declines, and that naive token-flip or alternative local pooling also degrades performance. In this broader sense, the empirical record supports the narrower LEAF-Mamba claim that explicit local extraction is a decisive complement to SSM backbones when locality is structurally important (Han et al., 2024).