---
title: 'LEAF-Mamba: RGB-D Salient Object Detection'
url: https://www.emergentmind.com/topics/leaf-mamba
type: topic
---

# LEAF-Mamba: RGB-D Salient Object Detection

LEAF-Mamba is a state space model-based network for RGB-D salient object detection whose name expands to “Local Emphatic and Adaptive Fusion.” It is designed to identify the most conspicuous objects in a scene by combining RGB appearance with depth cues, while addressing two specific deficiencies that arise when Mamba-style state space models are transferred directly to RGB-D saliency: weakened local semantics after 2D-to-1D scanning and inadequate cross-modality fusion. The model therefore couples a dual-stream VMamba-T encoder–decoder with a Local Emphatic State Space Module (LE-SSM) for multi-scale local dependency modeling and an Adaptive Fusion Module (AFM) for similarity-aware RGB–depth interaction and integration. In reported experiments it achieves state-of-the-art performance on seven RGB-D benchmarks and also generalizes to RGB-T salient object detection, with 84.5M parameters, 18.1G FLOPs, and 70.2 FPS [2509.18683].

## 1. Definition, task scope, and naming

In the published usage of the term, LEAF-Mamba denotes an RGB-D salient object detection model rather than a plant-disease recognizer. Its task is to predict a binary saliency map that highlights the most visually prominent objects in a scene by jointly exploiting an RGB image and a depth map. The model is motivated by the observation that RGB contributes color and texture information, whereas depth contributes geometric and spatial structure such as distance, shape, and foreground–background separation [2509.18683].

A recurrent misconception is that the label “LEAF-Mamba” necessarily refers to leaf-image classification because the same expression can also function as a descriptive family name for Mamba-based plant-leaf models. In particular, Mam-App has been described as “essentially a LEAF-Mamba model” in the sense of a Mamba-based, ultra-lightweight classifier for plant leaf diseases. This suggests that “LEAF-Mamba” has both a formal use—the RGB-D salient object detector—and a broader descriptive use for leaf-oriented Mamba architectures in adjacent literature [2601.21307].

The defining contribution of the RGB-D LEAF-Mamba system is not merely the substitution of attention with an SSM. Its distinctiveness lies in the combination of local emphatic modeling and adaptive fusion: local windowed selective scans restore fine 2D semantics, while fusion is modulated by cross-modality similarity and complementarity rather than by modality-symmetric aggregation [2509.18683].

## 2. Problem formulation and design rationale

RGB-D salient object detection requires simultaneous modeling of local contours, object-scale structure, and cross-modal agreement or disagreement. LEAF-Mamba is situated against two established limitations. First, CNN-based RGB-D detectors are constrained by local receptive fields and therefore struggle with true long-range dependencies. Second, Transformer-based detectors obtain global interactions through self-attention but incur quadratic complexity in the token count, which is especially burdensome in dual-modality settings [2509.18683].

The architectural rationale follows the Mamba state-space formulation. In continuous time, the underlying SSM is written as
$$
\begin{aligned}
h'(t) &= \bm{A} h(t) + \bm{B} x(t), \\
y(t) &= \bm{C} h(t) + \bm{D} x(t),
\end{aligned}
$$
and after discretization,
$$
\begin{aligned}
\overline{\bm{A}} &= \exp(\Delta \bm{A}), \\
\overline{\bm{B}} &\approx \Delta \bm{B}, \\
h_t &= \overline{\bm{A}} h_{t-1} + \overline{\bm{B}} x_t, \\
y_t &= \bm{C} h_t + \bm{D} x_t.
\end{aligned}
$$
In this form, Mamba supplies input-adaptive selective scanning with linear complexity in sequence length. LEAF-Mamba adopts this efficiency argument but adds mechanisms intended to correct two task-specific failures of direct SSM application: deficient local semantics after row/column-style serialization and insufficiently discriminative RGB–depth fusion [2509.18683].

The model’s central claim is therefore narrower than a general endorsement of SSMs. It proposes that RGB-D saliency requires both long-range dependency modeling and explicit restoration of local 2D proximity, together with fusion rules that distinguish complementary from reliable regions. This dual emphasis is what gives the acronym “LEAF” its architectural meaning.

## 3. Global architecture

LEAF-Mamba uses a dual-stream encoder–decoder architecture. The inputs are an RGB image and a depth map, both resized to \(256 \times 256\). The encoder consists of two separate VMamba-T streams, one for RGB and one for depth, and each stage produces multi-scale features
\(\{F^r_i\}_{i=1}^4\) and \(\{F^d_i\}_{i=1}^4\). In each stage, the last VMamba block is replaced by the Local Emphatic State Space Module, so that the stage outputs are already locally enriched before cross-modal processing [2509.18683].

At every scale, the corresponding RGB and depth features are passed to the Adaptive Fusion Module. AFM has two outputs. First, it generates inter-enhanced modality-specific features \(F'^r_i\) and \(F'^d_i\), which are propagated to the next encoder stage. Second, it produces fused RGB-D features \(F^f_i\), which are sent to the decoder. This organization makes fusion progressive rather than terminal: cross-modality interaction occurs repeatedly throughout the hierarchy instead of only after independent feature extraction has completed [2509.18683].

The decoder is described as an SSM-based FPN-like decoder. It takes the fused multi-scale features \(\{F^f_i\}_{i=1}^4\), applies up-sampling, VMamba blocks, and CBAM for refinement, and produces multi-level saliency maps \(\{P_i\}_{i=1}^4\) under deep supervision. The final output is \(P_1\). This decoder choice reflects a standard top-down saliency design, but the token-mixing and refinement remain SSM-centered rather than attention-centered [2509.18683].

A compact summary of the network is given below.

| Component | Role | Output |
|---|---|---|
| Dual VMamba-T encoder | Separate RGB and depth feature extraction | \(\{F^r_i\}, \{F^d_i\}\) |
| LE-SSM | Multi-scale local semantic enhancement | Local-enhanced stage features |
| AFM | Cross-modality interaction and fusion | \(F'^r_i, F'^d_i, F^f_i\) |
| SSM-based FPN decoder | Multi-scale saliency reconstruction | \(\{P_i\}\), final \(P_1\) |

The overall design preserves the linear-complexity appeal of Mamba while inserting specialized modules exactly where the base VMamba backbone would otherwise be least adapted to RGB-D saliency.

## 4. Local Emphatic State Space Module

LE-SSM is the mechanism by which LEAF-Mamba restores local semantics. It is a single-branch block with two residual modules, structurally analogous to a Transformer block, but its key novelty is the replacement of the standard SS2D block with a multi-scale windowed 2D selective scan, denoted MSW-SS2D. Given a feature map \(X \in \mathbb{R}^{H \times W \times C}\), LE-SSM defines four traversal paths:
\(\mathrm{H_1}\), \(\mathrm{HF_2}\), \(\mathrm{V_4}\), and \(\mathrm{VF_8}\), corresponding respectively to horizontal, flipped horizontal, vertical, and flipped vertical scans with window sizes 1, 2, 4, and 8 [2509.18683].

Each path unfolds the feature map according to its direction and windowing, applies an S6 selective SSM independently, and reshapes the result back to 2D. The outputs are aggregated as
$$
Y = \sum_{\mathrm{Scan} \in \mathcal{S}} \mathrm{Reshape}(\mathrm{S6}(\mathrm{Scan}(X))),
$$
where \(\mathcal{S} = \{\mathrm{H_1}, \mathrm{HF_2}, \mathrm{V_4}, \mathrm{VF_8}\}\). The operational point is that small windows capture very local detail and larger windows capture broader local neighborhoods, while the four directions reduce orientation bias [2509.18683].

This module is intended to correct a specific weakness of vanilla VMamba-style scanning: flattening a 2D map into a 1D sequence can break adjacency relations and diminish neighborhood modeling. LE-SSM reintroduces 2D-aware locality without abandoning selective scanning. The paper states that the total operations remain comparable to VMamba’s SS2D, so the local enhancement is not presented as a cost-increasing attention substitute but as a scanning redesign [2509.18683].

The ablation evidence is explicit. On NJUD and SSD, MSW-SS2D outperforms three alternatives—SS2D, continuous scan, and fixed windowed scan. The reported comparison against SS2D is:
- NJUD: \(F_{\beta}=0.917\), MAE \(=0.030\) for SS2D versus \(F_{\beta}=0.931\), MAE \(=0.028\) for MSW-SS2D.
- SSD: \(F_{\beta}=0.851\), MAE \(=0.044\) for SS2D versus \(F_{\beta}=0.872\), MAE \(=0.039\) for MSW-SS2D.

These numbers support the interpretation that local emphatic scanning improves both object quality and boundary accuracy, particularly on smaller or more structurally delicate targets [2509.18683].

## 5. Adaptive Fusion Module

AFM is the cross-modality core of LEAF-Mamba. It is built around three subcomponents: Cross-modality Second-order Pooling (CSoP), the Selective Interaction Module (SIM), and the Selective Enhancement Module (SEM). The module’s purpose is twofold: to promote complementary interaction where RGB and depth differ, and to strengthen reliable integration where they agree [2509.18683].

CSoP computes modality-specific similarity maps from second-order statistics rather than from pointwise similarity. For same-scale RGB and depth features \(F^r\) and \(F^d\), a \(1 \times 1\) convolution and downsampling reduce them to \((H,W,C)=(8,8,96)\), after which they are flattened into tokens
\(T^r, T^d \in \mathbb{R}^{HW \times C}\). A global cross-covariance matrix is then formed:
$$
M_{i,j} = \mathrm{Cov}(T^d_i, T^r_j).
$$
Column-wise convolution and projection yield the RGB similarity vector, and row-wise convolution and projection yield the depth similarity vector; after reshaping and upsampling, these become modality-specific similarity maps \(S^r\) and \(S^d\) aligned to the original resolution [2509.18683].

SIM uses the complementary regions, encoded by distance maps
$$
D^r = 1 - S^r, \qquad D^d = 1 - S^d,
$$
to drive cross-modality interaction inside the SSM itself. The distinctive operation is the swapping and weighting of the projection matrix \(\bm{C}\). For RGB,
$$
\begin{aligned}
\overline{\bm{A}^r} &= \exp(\Delta^r \bm{A}^r), \\
\overline{\bm{B}^r} &= \Delta^r \bm{B}^r, \\
h^r_t &= \overline{\bm{A}^r} h^r_{t-1} + \overline{\bm{B}^r} X^r_t, \\
Y^r_t &= (D^r \bm{C}^d) h^r_t + \bm{D}^r X^r_t,
\end{aligned}
$$
and an analogous equation holds for depth with \(\bm{C}^r\). The effect is cross-attention-like, but realized through the SSM output projection rather than through explicit attention matrices [2509.18683].

SEM then refines single-modality reliability. It computes new similarity maps \(S'^r\) and \(S'^d\) from the interacted features and weights the branch-specific projection matrices by those maps:
$$
Y^{\prime r}_t = (S^{\prime r} \bm{C}^{\prime r}) h^{\prime r}_t + \bm{D}^{\prime r} X^{\prime r}_t,
$$
with a symmetric expression for depth. In other words, SIM emphasizes disagreement as a cue for complementary exchange, whereas SEM emphasizes agreement as a cue for trustworthy enhancement [2509.18683].

The paper does not write an explicit closed-form equation for the final fusion \(F^f\). A plausible interpretation is that fusion is a learned combination of the reliability-enhanced RGB and depth features after SIM and SEM, but that extrapolation should be distinguished from the explicit module definitions. The ablations nevertheless identify the functional hierarchy clearly: CSoP improves both SIM and SEM, covariance-based similarity is reported as better than cosine similarity, swapping \(\bm{C}\) is more effective than swapping inputs or \(\bm{B}\), and weighting \(\bm{C}\) is reported as the most effective SEM design [2509.18683].

## 6. Training protocol, benchmarks, and empirical profile

LEAF-Mamba is trained and evaluated on seven RGB-D salient object detection datasets: NJUD (1985 pairs), NLPR (1000), STERE (1000), SIP (929 high-res), SSD (80), LFSD (100), and DUT-D (1200). Following A2dele and TriTransNet, the training split uses 1485 NJUD + 700 NLPR + 800 DUT-D, while testing uses the remaining NJUD, NLPR, and DUT-D samples together with all STERE, SIP, SSD, and LFSD images. For RGB-T evaluation, the datasets are VT821, VT1000, and VT5000; training uses 2500 images from VT5000, and testing uses the remaining VT5000 images plus all VT821 and VT1000 samples [2509.18683].

The implementation uses PyTorch with a VMamba-T backbone pre-trained on ImageNet-1K. Inputs are \(256 \times 256\), the optimizer is Adam, batch size is 8, the initial learning rate is \(1 \times 10^{-4}\), the learning rate is decayed by a factor of 10 every 60 epochs, and training runs for 200 epochs with random flipping and random rotation. The loss is the pixel position-aware loss from F3Net with multi-level supervision over the decoder outputs [2509.18683].

The reported efficiency profile is unusually strong for an RGB-D saliency model of this scale. LEAF-Mamba uses 84.5M parameters, 18.1G FLOPs, and runs at 70.2 FPS. Among the listed comparison methods, 18.1G is the lowest FLOP count and 70.2 is the highest FPS. The paper contrasts this with several Transformer-based models, including DCT at 49.0G FLOPs and 80M parameters, CAT at 341.8G FLOPs, 262.6M parameters, and 11 FPS, and HFMD at 242.2G FLOPs, 431.6M parameters, and 9 FPS [2509.18683].

The quantitative RGB-D results are reported in terms of \(F_{\beta}\), \(S_{\alpha}\), \(E_{\xi}\), and MAE \(M\). LEAF-Mamba records:
- **NJUD**: \(F_{\beta}=0.945\), \(S_{\alpha}=0.940\), \(E_{\xi}=0.967\), MAE \(=0.025\).
- **NLPR**: \(F_{\beta}=0.939\), \(S_{\alpha}=0.945\), \(E_{\xi}=0.976\), MAE \(=0.016\).
- **SIP**: \(F_{\beta}=0.935\), \(S_{\alpha}=0.920\), \(E_{\xi}=0.950\), MAE \(=0.032\).
- **STERE**: \(F_{\beta}=0.935\), \(S_{\alpha}=0.933\), \(E_{\xi}=0.958\), MAE \(=0.026\).
- **SSD**: \(F_{\beta}=0.904\), \(S_{\alpha}=0.918\), \(E_{\xi}=0.953\), MAE \(=0.033\).
- **LFSD**: \(F_{\beta}=0.908\), \(S_{\alpha}=0.907\), \(E_{\xi}=0.934\), MAE \(=0.045\).
- **DUT-D**: \(F_{\beta}=0.958\), \(S_{\alpha}=0.952\), \(E_{\xi}=0.973\), MAE \(=0.019\).

On these datasets it consistently matches or exceeds the strongest published comparators. On SSD, for example, the MAE of 0.033 is reported as an approximately 13.2% reduction versus VST++, and on LFSD the MAE of 0.045 is reported as an approximately 8.16% reduction versus the second-best method [2509.18683].

## 7. Generalization, limitations, and place within the Mamba literature

LEAF-Mamba extends directly to RGB-T salient object detection by treating the thermal image as the second modality without architectural changes. On VT821 it reports \(F_{\beta}=0.885\), \(S_{\alpha}=0.926\), \(E_{\xi}=0.943\), and MAE \(=0.020\), all described as best; on VT5000 it reports \(F_{\beta}=0.893\), \(S_{\alpha}=0.919\), \(E_{\xi}=0.958\), and MAE \(=0.021\), where \(E_{\xi}\) is best and the other three metrics are second best to ConTriNet. This cross-domain transfer is presented as evidence that the AFM design generalizes beyond depth to thermal fusion [2509.18683].

The limitations are specific rather than generic. Failure cases are reported when RGB and depth are both severely degraded, leaving little reliable information for the model to exploit. Additional difficult cases include transparent objects, such as glass, and strongly occluded objects, such as partially hidden kites. These cases are consistent with the model’s reliance on meaningful modality-specific similarity and complementarity maps: when both modalities are unreliable, adaptive fusion has little stable structure to amplify [2509.18683].

Within the broader Mamba literature, LEAF-Mamba exemplifies a particular design trend: the retention of linear-time SSM scanning together with explicit recovery of local structure that would otherwise be weakened by serialization. Related work in lightweight medical segmentation uses Residual Vision Mamba Layers to model long-range spatial dependencies with linear complexity and reports 116x and 21x reductions in parameter and computation costs relative to nnU-Net on a 2D benchmark [2403.05246]. In a different domain, Mam-App is described as a Mamba-based, ultra-lightweight visual classifier that can be understood as a LEAF-Mamba-style model for plant leaf disease classification, achieving 99.58% accuracy with 0.051M parameters on the PlantVillage Apple Leaf Disease dataset [2601.21307].

Taken together, these comparisons indicate that LEAF-Mamba is best understood not as a generic synonym for any Mamba vision model, but as a specific RGB-D architecture whose defining ideas are local emphatic scanning and adaptive cross-modality fusion. The broader literature suggests that these ideas belong to a wider methodological movement in which Mamba-based systems seek to preserve long-range modeling while reintroducing locality, modality structure, or domain priors in task-specific ways.

Source: https://www.emergentmind.com/topics/leaf-mamba