DeSamba: Sparse Neural Architectures

• DeSamba is a class of neural architectures that combine sparse token selection with frequency-adaptive modulation to process high-dimensional spatial and spectral data.
• It integrates domain-specific modules for hyperspectral image classification and multi-sequence MRI lesion detection using hybrid CNN and state-space models.
• Empirical evaluations show that DeSamba reduces computational costs while enhancing accuracy and efficiency in complex imaging scenarios.

DeSamba refers to a class of neural network architectures that leverage sparse, structured, and frequency-adaptive mechanisms for high-dimensional data classification, particularly in hyperspectral imaging (HSI) and multi-sequence 3D medical imaging. Two prominent DeSamba frameworks have been established for distinct domains: hyperspectral image classification (Xu et al., 13 Apr 2025) and multi-sequence MRI lesion classification (Wang et al., 21 Jul 2025). Both frameworks share a focus on adaptive spatial-spectral modeling but incorporate domain-specific innovations in sequence selection, representation decoupling, and frequency-domain adaptation.

1. Motivation and Problem Context

DeSamba frameworks address the need for efficient and detail-preserving modeling of high-dimensional data in contexts where both spatial and spectral/frequency information are essential. In hyperspectral image classification, dense sequence modeling incurs high computational cost and may dilute important small-class details. In multi-sequence 3D MRI, simple data fusion is insufficient, as shared anatomical semantics and sequence-specific contrast signatures must be separated to capture pathological variation. Both application domains require models that:

• Select and adaptively process salient spatial and spectral (or frequency) components.
• Reduce computational burden through principled sparsity.
• Explicitly fuse or decouple shared and unique contextual features for robust classification.

2. Core Methodologies

Sparse Deformable Mamba (SDMamba) for Hyperspectral Images

SDMamba (also referred to as DeSamba in this context) employs Sparse Deformable Sequencing (SDS) to efficiently select a subset of tokens from the spatial or spectral domain. Given token representations $Z \in \mathbb{R}^{N \times C}$ and an anchor vector $z_{\text{anchor}} \in \mathbb{R}^C$, a relevance score vector $s \in \mathbb{R}^N$ is computed using cosine similarity:

$$s_i = \frac{z_{\rm anchor}^\top z_i}{\|z_{\rm anchor}\| \|z_i\|}$$

Tokens with the top $M = \lfloor \lambda N \rfloor$ scores are selected using either a hard mask or a soft-top-$k$ relaxation, resulting in a sparse, deformable sequence $\bar Z$ whose composition and order are learned.

Spatial and spectral information are processed by dedicated modules:

• SDSpaM (Sparse Deformable Spatial Mamba Module) processes spatial tokens.
• SDSpeM (Sparse Deformable Spectral Mamba Module) processes spectral tokens.

Each module includes a stem convolution, tokenization, SDS-based sparse selection, a stack of Mamba state-space model blocks, and residual scattering to reconstruct full patch representations.

Feature fusion occurs via attention: for feature maps $F_{\rm spa}$ and $F_{\rm spe}$, query/key/value projections are computed, followed by an attention-weighted fusion that integrates spatial and spectral cues:

$$\mathrm{Scores} = \frac{Q K^\top}{\sqrt{d_k}},\quad \mathrm{Attn} = \mathrm{softmax}(\mathrm{Scores}),\quad F_{\text{fused}} = \mathrm{Attn} \cdot V$$

DeSamba for Multi-Sequence 3D MRI

In 3D MRI, DeSamba denotes a framework that combines spectral adaptive processing with explicit sequence decoupling:

1. Multi-sequence encoders for each MRI sequence operate in parallel: a ConvNeXtV2-derived SAMNet enhanced with Spectral Adaptive Modulation Blocks (SAMB) and a state-space model backbone (MambaOut).
2. Decoupled Representation Learning Module (DRLM) separates each sequence's feature $z_{\text{anchor}} \in \mathbb{R}^C$0 into unique ($z_{\text{anchor}} \in \mathbb{R}^C$1) and shared ($z_{\text{anchor}} \in \mathbb{R}^C$2) components. These are enforced by self-reconstruction and cross-reconstruction objectives:

$z_{\text{anchor}} \in \mathbb{R}^C$3

$z_{\text{anchor}} \in \mathbb{R}^C$4

$z_{\text{anchor}} \in \mathbb{R}^C$5

Losses include:

$z_{\text{anchor}} \in \mathbb{R}^C$6

Overall loss combines classification and reconstruction terms.

1. SAMB applies 3D FFT-based frequency analysis to each input patch. After computing real ($z_{\text{anchor}} \in \mathbb{R}^C$7) and imaginary ($z_{\text{anchor}} \in \mathbb{R}^C$8) FFT channels and magnitude $z_{\text{anchor}} \in \mathbb{R}^C$9, spectral features are adaptively modulated by learned factors before inverse FFT transforms reconstruct the spatial domain. Spatial and frequency branch outputs are gated and fused, with a residual added to the input.

3. Training Protocols and Hyperparameters

Both variants of DeSamba utilize end-to-end supervised training, with configuration tailored to data dimensionality:

System	Optimizer	Batch Size	Learning Rate	Epochs	Weight Decay	Backbone Channels	Key Hyperparameters
SDMamba (HSI)	Adam	64	$s \in \mathbb{R}^N$0	100	$s \in \mathbb{R}^N$1	$s \in \mathbb{R}^N$2	$s \in \mathbb{R}^N$3 (sparsity), patch size 9/13
DeSamba (MRI)	AdamW	4	$s \in \mathbb{R}^N$4	200	$s \in \mathbb{R}^N$5	Channel counts per backbone	$s \in \mathbb{R}^N$6 (loss weights), 308 GFLOPs

SDMamba uses standard cross-entropy loss for center-pixel label classification. DeSamba (MRI) utilizes a composite loss function including cross-entropy, self-reconstruction, and cross-reconstruction terms.

4. Evaluation and Empirical Findings

Experiments on HSI benchmarks (Indian Pines, Pavia U) and medical imaging datasets (spinal metastasis, spondylitis MRI) reveal that DeSamba frameworks outperform contemporary state-of-the-art baselines in both accuracy and efficiency.

SDMamba Performance

• Indian Pines (10% train): OA 99.44%, AA 99.36%, Kappa 99.36% (vs. next best OA 99.18%)
• Pavia U (3% train): OA 99.14%, AA 98.45%, Kappa 98.85% (vs. next best OA 99.00%)
• ≈ 59% FLOPs reduction using SDS ($s \in \mathbb{R}^N$7) vs. dense (172.4 MFLOPs vs. 416.2 MFLOPs)
• Accuracy is maximized at $s \in \mathbb{R}^N$8; increased sparsity does not harm overall classification

DeSamba (MRI) Performance

• Spinal Metastasis (external set n=372): Top-1 acc 62.10% (next best 55.11%), F1 63.62%, AUC 87.71%, Top-3 acc 93.55%
• Spondylitis (external): acc 64.52%, AUC 73.88% (next best 58.06%, 67.22%)
• Ablation: Adding spectral adaptation (SAMNet) or SSMs (MambaOut) to baseline improves accuracy. DRLM yields the largest boost, and the full model achieves the best results.

5. Architectural Significance and Innovations

Both DeSamba frameworks exemplify advances in adaptive sequence modeling:

• SDS/DRLM-based decoupling: Maximizes utility of both global shared context and sequence/patch-specific information, critical in both hyperspectral and multi-sequence medical imaging.
• Explicit frequency-domain modulation (SAMB): Enables targeted enhancement of discriminative frequency bands, aligning with observed lesion- or material-specific frequency signatures.
• Sparse token selection: Reduces computational load while preserving class-detail and discriminative power. In SDMamba, sparse deformable sequencing can be solved exactly (top-$s \in \mathbb{R}^N$9 selection), with an optional differentiable relaxation for end-to-end learning.
• Hybrid state-space and CNN backbones: Combining Mamba-based sequence mixing with convolutional processing enhances context modeling in both spatial and spectral domains.

6. Limitations and Prospects

The modularity of DeSamba frameworks indicates potential for extension to other high-dimensional, multi-modal imaging tasks. Limitations include dataset dependency (e.g., optimal sparsity $$s_i = \frac{z_{\rm anchor}^\top z_i}{\|z_{\rm anchor}\| \|z_i\|}$$0 may vary) and fixed architecture design (e.g., backbone choices, pre-defined fusion schemes). The original authors propose further research into automated frequency-band selection and adaptation of spectral decoupling mechanisms to other modalities (such as diffusion or perfusion MRI), as well as segmentation beyond classification (Xu et al., 13 Apr 2025, Wang et al., 21 Jul 2025).

7. Conclusions

DeSamba encompasses a family of architectures integrating sparsity, spectral adaptivity, and sequence decoupling to address challenges in complex high-dimensional data classification. By leveraging techniques such as Sparse Deformable Sequencing, Spectral Adaptive Modulation, and Decoupled Representation Learning, DeSamba achieves state-of-the-art accuracy and efficiency on both hyperspectral and multi-sequence 3D medical imaging tasks. The separation and targeted fusion of spatial and spectral/frequency features underpins its empirical performance and suggests broader applicability across high-dimensional multi-modal data analyses.