---
title: 'DeSamba: Sparse Neural Architectures'
url: https://www.emergentmind.com/topics/desamba
type: topic
---

# DeSamba: Sparse Neural Architectures

DeSamba refers to a class of neural network architectures that leverage sparse, structured, and frequency-adaptive mechanisms for high-dimensional data classification, particularly in hyperspectral imaging (HSI) and multi-sequence 3D medical imaging. Two prominent DeSamba frameworks have been established for distinct domains: hyperspectral image classification [2504.09446] and multi-sequence MRI lesion classification [2507.15487]. Both frameworks share a focus on adaptive spatial-spectral modeling but incorporate domain-specific innovations in sequence selection, representation decoupling, and frequency-domain adaptation.

## 1. Motivation and Problem Context

DeSamba frameworks address the need for efficient and detail-preserving modeling of high-dimensional data in contexts where both spatial and spectral/frequency information are essential. In hyperspectral image classification, dense sequence modeling incurs high computational cost and may dilute important small-class details. In multi-sequence 3D MRI, simple data fusion is insufficient, as shared anatomical semantics and sequence-specific contrast signatures must be separated to capture pathological variation. Both application domains require models that:

- Select and adaptively process salient spatial and spectral (or frequency) components.
- Reduce computational burden through principled sparsity.
- Explicitly fuse or decouple shared and unique contextual features for robust classification.

## 2. Core Methodologies

### Sparse Deformable Mamba (SDMamba) for Hyperspectral Images

SDMamba (also referred to as DeSamba in this context) employs Sparse Deformable Sequencing (SDS) to efficiently select a subset of tokens from the spatial or spectral domain. Given token representations $Z \in \mathbb{R}^{N \times C}$ and an anchor vector $z_{\text{anchor}} \in \mathbb{R}^C$, a relevance score vector $s \in \mathbb{R}^N$ is computed using cosine similarity:
$$
s_i = \frac{z_{\rm anchor}^\top z_i}{\|z_{\rm anchor}\| \|z_i\|}
$$
Tokens with the top $M = \lfloor \lambda N \rfloor$ scores are selected using either a hard mask or a soft-top-$k$ relaxation, resulting in a sparse, deformable sequence $\bar Z$ whose composition and order are learned.

Spatial and spectral information are processed by dedicated modules:
- SDSpaM (Sparse Deformable Spatial Mamba Module) processes spatial tokens.
- SDSpeM (Sparse Deformable Spectral Mamba Module) processes spectral tokens.

Each module includes a stem convolution, tokenization, SDS-based sparse selection, a stack of Mamba state-space model blocks, and residual scattering to reconstruct full patch representations.

Feature fusion occurs via attention: for feature maps $F_{\rm spa}$ and $F_{\rm spe}$, query/key/value projections are computed, followed by an attention-weighted fusion that integrates spatial and spectral cues:
$$
\mathrm{Scores} = \frac{Q K^\top}{\sqrt{d_k}},\quad
\mathrm{Attn} = \mathrm{softmax}(\mathrm{Scores}),\quad
F_{\text{fused}} = \mathrm{Attn} \cdot V
$$

### DeSamba for Multi-Sequence 3D MRI

In 3D MRI, DeSamba denotes a framework that combines spectral adaptive processing with explicit sequence decoupling:

1. **Multi-sequence encoders** for each MRI sequence operate in parallel: a ConvNeXtV2-derived SAMNet enhanced with Spectral Adaptive Modulation Blocks (SAMB) and a state-space model backbone (MambaOut).
2. **Decoupled Representation Learning Module (DRLM)** separates each sequence's feature $f_i$ into unique ($U_i$) and shared ($S_{ij}$) components. These are enforced by self-reconstruction and cross-reconstruction objectives:
   $$
   U_i = \mathrm{Enc}_u(f_i),\qquad S_{ij} = \mathrm{Enc}_s(f_i, f_j)
   $$
   $$
   \mathrm{SRF}_i = \mathrm{SDec}_i(U_i, [S_{ij}]_{j \neq i})
   $$
   $$
   \mathrm{CRF}_i = \mathrm{CDec}_i(U_i, [S_{jk}]_{j,k \neq i})
   $$
   Losses include:
   $$
   \mathcal{L}_{\text{self}} = \sum_i \| \mathrm{SRF}_i - f_i \|_1,\qquad
   \mathcal{L}_{\text{cross}} = \sum_i \| \mathrm{CRF}_i - f_i \|_1
   $$
   Overall loss combines classification and reconstruction terms.
3. **SAMB** applies 3D FFT-based frequency analysis to each input patch. After computing real ($R$) and imaginary ($I$) FFT channels and magnitude $\phi$, spectral features are adaptively modulated by learned factors before inverse FFT transforms reconstruct the spatial domain. Spatial and frequency branch outputs are gated and fused, with a residual added to the input.

## 3. Training Protocols and Hyperparameters

Both variants of DeSamba utilize end-to-end supervised training, with configuration tailored to data dimensionality:

| System              | Optimizer | Batch Size | Learning Rate    | Epochs | Weight Decay | Backbone Channels | Key Hyperparameters                                |
|---------------------|-----------|------------|------------------|--------|--------------|-------------------|----------------------------------------------------|
| SDMamba (HSI)       | Adam      | 64         | $1\times10^{-4}$ | 100    | $1\times10^{-5}$ | $D=256$           | $\lambda=0.30$ (sparsity), patch size 9/13         |
| DeSamba (MRI)       | AdamW     | 4          | $1\times10^{-4}$ | 200    | $1\times10^{-5}$ | Channel counts per backbone | $\alpha=\beta=0.5$ (loss weights), 308 GFLOPs      |

SDMamba uses standard cross-entropy loss for center-pixel label classification. DeSamba (MRI) utilizes a composite loss function including cross-entropy, self-reconstruction, and cross-reconstruction terms.

## 4. Evaluation and Empirical Findings

Experiments on HSI benchmarks (Indian Pines, Pavia U) and medical imaging datasets (spinal metastasis, spondylitis MRI) reveal that DeSamba frameworks outperform contemporary state-of-the-art baselines in both accuracy and efficiency.

### SDMamba Performance

- **Indian Pines (10% train):** OA 99.44%, AA 99.36%, Kappa 99.36% (*vs.* next best OA 99.18%)
- **Pavia U (3% train):** OA 99.14%, AA 98.45%, Kappa 98.85% (*vs.* next best OA 99.00%)
- *≈ 59% FLOPs reduction* using SDS ($\lambda=0.30$) vs. dense (172.4 MFLOPs vs. 416.2 MFLOPs)
- Accuracy is maximized at $\lambda=0.30$; increased sparsity does not harm overall classification

### DeSamba (MRI) Performance

- **Spinal Metastasis (external set n=372):** Top-1 acc 62.10% (next best 55.11%), F1 63.62%, AUC 87.71%, Top-3 acc 93.55%
- **Spondylitis (external):** acc 64.52%, AUC 73.88% (next best 58.06%, 67.22%)
- Ablation: Adding spectral adaptation (SAMNet) or SSMs (MambaOut) to baseline improves accuracy. DRLM yields the largest boost, and the full model achieves the best results.

## 5. Architectural Significance and Innovations

Both DeSamba frameworks exemplify advances in adaptive sequence modeling:

- **SDS/DRLM-based decoupling:** Maximizes utility of both global shared context and sequence/patch-specific information, critical in both hyperspectral and multi-sequence medical imaging.
- **Explicit frequency-domain modulation (SAMB):** Enables targeted enhancement of discriminative frequency bands, aligning with observed lesion- or material-specific frequency signatures.
- **Sparse token selection:** Reduces computational load while preserving class-detail and discriminative power. In SDMamba, sparse deformable sequencing can be solved exactly (top-$M$ selection), with an optional differentiable relaxation for end-to-end learning.
- **Hybrid state-space and CNN backbones:** Combining Mamba-based sequence mixing with convolutional processing enhances context modeling in both spatial and spectral domains.

## 6. Limitations and Prospects

The modularity of DeSamba frameworks indicates potential for extension to other high-dimensional, multi-modal imaging tasks. Limitations include dataset dependency (e.g., optimal sparsity $\lambda$ may vary) and fixed architecture design (e.g., backbone choices, pre-defined fusion schemes). The original authors propose further research into automated frequency-band selection and adaptation of spectral decoupling mechanisms to other modalities (such as diffusion or perfusion MRI), as well as segmentation beyond classification [2504.09446, 2507.15487].

## 7. Conclusions

DeSamba encompasses a family of architectures integrating sparsity, spectral adaptivity, and sequence decoupling to address challenges in complex high-dimensional data classification. By leveraging techniques such as Sparse Deformable Sequencing, Spectral Adaptive Modulation, and Decoupled Representation Learning, DeSamba achieves state-of-the-art accuracy and efficiency on both hyperspectral and multi-sequence 3D medical imaging tasks. The separation and targeted fusion of spatial and spectral/frequency features underpins its empirical performance and suggests broader applicability across high-dimensional multi-modal data analyses.

Source: https://www.emergentmind.com/topics/desamba