Spectral-Informed Mamba

• Spectral-Informed Mamba is an advanced architecture that extends linear state space models by incorporating spectral priors and traversal strategies for improved point cloud processing.
• It leverages Laplacian eigendecomposition and methods like SAST and HLT to create isometry-invariant traversals that enhance data ordering and segmentation accuracy.
• Empirical results demonstrate consistent 1–2% performance gains in classification, segmentation, and reconstruction compared to Transformer- and Mamba-based baselines.

Spectral-Informed Mamba encompasses a class of architectures that extend the Mamba linear state space model (SSM) paradigm by incorporating “spectral” priors and traversal strategies into the core sequence modeling process. Originally proposed to address limitations in Mamba’s handling of spatially structured, permutation-sensitive data—especially point clouds in 3D space—Spectral-Informed Mamba leverages the spectrum of graph Laplacians, spectral transforms, and invariant traversals to improve viewpoint robustness, shape manifold capture, and information ordering. This methodology has demonstrated advantages in point cloud classification, segmentation, and self-supervised reconstruction, yielding consistent gains over both Transformer- and Mamba-based state-of-the-art baselines (Bahri et al., 6 Mar 2025).

1. Spectral Graph Theory and Laplacian Foundations

At the heart of the Spectral-Informed Mamba (SI-Mamba) is the construction of an undirected, weighted "patch-connectivity" graph defined over local patches of the data. For $N_c$ patches, the adjacency matrix $W \in \mathbb{R}^{N_c \times N_c}$ is formed via Gaussian similarity,

$$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$

The diagonal degree matrix $D$ has $D_{ii} = \sum_j W_{ij}$. The random walk Laplacian is then $L_{rw} = I - D^{-1} W$, a symmetric positive semidefinite matrix with eigendecomposition $L_{rw} = V \Lambda V^\top$, where the eigenvectors $v^{(1)},\dots,v^{(N_c)}$ correspond to increasing non-negative eigenvalues. The lowest-frequency (constant) eigenvector is discarded, and the next $s$ nonzero modes $\{v^{(k)}\}_{k=2}^{s+1}$ encode the essential patch connectivity for downstream traversal and partitioning.

This Laplacian eigendecomposition enables SI-Mamba to construct traversal orders and partitions that are strictly isometry-invariant, robust to viewing transformations, and sensitive to manifold topology (Bahri et al., 6 Mar 2025).

2. Isometry-Invariant Traversal: Spectral-Aware Sequential Traversal (SAST)

To establish an ordering of graph patches that is canonical up to permutations and sign ambiguity, SI-Mamba employs the following spectral-processing and ordering protocol:

• For each eigenmode $W \in \mathbb{R}^{N_c \times N_c}$0, flip sign if $W \in \mathbb{R}^{N_c \times N_c}$1.
• If $W \in \mathbb{R}^{N_c \times N_c}$2 and $W \in \mathbb{R}^{N_c \times N_c}$3, swap modes for stable ordering.
• For each mode $W \in \mathbb{R}^{N_c \times N_c}$4:
  • Arrange patch indices $W \in \mathbb{R}^{N_c \times N_c}$5 by sorting $W \in \mathbb{R}^{N_c \times N_c}$6 in increasing (forward) and decreasing (reverse) order, yielding $W \in \mathbb{R}^{N_c \times N_c}$7 permutations $W \in \mathbb{R}^{N_c \times N_c}$8 and $W \in \mathbb{R}^{N_c \times N_c}$9.

Each Mamba block processes the token sequence $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$0 times, once per traversal, with the outputs concatenated. This approach aligns the sequence model’s processing path with the intrinsic spectral geometry of the patch graph, providing equivariance to spatial transformations and view alterations, and superior manifold unfolding compared to grid- or radius-based traversals (Bahri et al., 6 Mar 2025).

3. Recursive Spectral Patch Partitioning (HLT)

For segmentation, a hierarchical, Laplacian-informed traversal (HLT) is constructed via recursive binary partitioning:

• For each patch $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$1 and mode $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$2, threshold $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$3 at its mean to produce a bit $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$4.
• The per-patch bitstrings $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$5 are collapsed into an integer $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$6 via binary-to-integer conversion.
• Patches are then sorted by $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$7, creating an ordering $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$8 that recursively splits the graph into $$W_{ij} = \begin{cases} \exp(-\|p_{s_i} - p_{s_j}\|^2/\sigma) & \text{if } j \in K\mathsf{NN}(i) \text{ or } i \in K\mathsf{NN}(j)\ 0 & \text{otherwise}. \end{cases}$$9 coherent groups (with $D$0 yielding up to 16 groups for 128 patches).
• Within-group tie-breaking uses randomness or the first eigenmode.

Pseudocode for binary partitioning is provided in the original text. This partitioning strategy produces traversal orders well-matched to segment boundaries and shape part structure due to its dependence on the smooth spectral modes of the Laplacian (Bahri et al., 6 Mar 2025).

4. Token Placement in Masked Autoencoding (TAR)

In the self-supervised Masked Autoencoder (MAE) paradigm, token masking and restoration must respect the canonical traversal defined by SAST or HLT to preserve intrinsic ordering:

• A random subset $D$1 of tokens is masked, their traversal positions $D$2 recorded.
• Tokens are removed, and remaining ones input to the encoder.
• Before decoding, masked tokens are re-inserted at positions $D$3, with mask tokens at the masked locations and encoder outputs at others.
• The decoder reconstructs the original point sets $D$4, with loss

$D$5

This pipeline ensures consistency of the input/output order for Mamba (unlike naive masking), preserves spatial and spectral context, and empirically improves reconstruction and representation learning (Bahri et al., 6 Mar 2025).

5. Framework Architecture and Data Flow

The SI-Mamba processing pipeline comprises:

• Point cloud sampling (N-points with Farthest Point Sampling $D$6 $D$7 patch centers, then KNN for $D$8 local patches).
• Graph Laplacian computation and eigendecomposition.
• Task-dependent traversal:
  • Classification: Apply SAST, process $D$9 traversals in Mamba blocks, concatenate outputs, classify via MLP.
  • Segmentation: Apply HLT traversal, Mamba blocks generate per-patch features, upsample via nearest-neighbor interpolation, per-point softmax.
  • Self-supervised: Apply TAR in the chosen traversal, Mamba encoder/decoder reconstructs masked patches.
• All Mamba modules use the efficient SSM implementation with input-adaptive parameter selection as in [Gu & Dao, 2023].

This unified workflow enables SI-Mamba to handle variable tasks and modalities while benefiting systematically from spectral ordering and adaptation (Bahri et al., 6 Mar 2025).

6. Training Configurations and Hyperparameters

For all experiments, training uses:

• Cross-entropy loss for classification/segmentation, reconstruction loss for MAE (as above).
• No explicit spectral regularizers.
• Standard hyperparameters: $D_{ii} = \sum_j W_{ij}$0 patches, $D_{ii} = \sum_j W_{ij}$1 points/patch, $D_{ii} = \sum_j W_{ij}$2 neighbors, $D_{ii} = \sum_j W_{ij}$3 spectral modes, mask ratio $D_{ii} = \sum_j W_{ij}$4, Laplacian bandwidth $D_{ii} = \sum_j W_{ij}$5, Mamba hidden size $D_{ii} = \sum_j W_{ij}$6, 12 layers, batch size 32, AdamW optimizer.
• Learning rate $D_{ii} = \sum_j W_{ij}$7 with 10-epoch warmup and decay to $D_{ii} = \sum_j W_{ij}$8; weight decay $D_{ii} = \sum_j W_{ij}$9, 200 total epochs.

These design and optimization settings are fixed across all examined tasks, ensuring robustness of reported findings (Bahri et al., 6 Mar 2025).

7. Empirical Results and Significance

SI-Mamba achieves state-of-the-art or superior performance across three core point cloud tasks:

• Few-shot classification (ModelNet40): SI-Mamba matches or slightly surpasses Point-M2AE and Point-MAE (e.g., 98.6% vs 98.4% at 5-way 20-shot).
• Part segmentation (ShapeNetPart, mIoU): Training from scratch, SI-Mamba (HLT) yields 85.9% vs 85.7% (Point-MAE) and 85.8% (Point-Mamba); with pretrained backbones, performance is comparable or marginally below Transformer-M2AE.
• Real-world object classification (ScanObjectNN): SI-Mamba achieves 92.25% (scratch) vs 90.87% (Point-Mamba), and 94.32% (pretrained) vs 91.22% (Point-M2AE), 92.77% (Point-MAE).

Across tasks, the adoption of spectral-informed traversals (SAST and HLT) and MAE token restoration delivers consistent 1–2 percentage point absolute improvements over traditional and contemporary baselines (Bahri et al., 6 Mar 2025).

---

References

• Spectral Informed Mamba for Robust Point Cloud Processing (Bahri et al., 6 Mar 2025)