---
title: Multi-Prior Hierarchical Mamba Network
url: https://www.emergentmind.com/topics/multi-prior-hierarchical-mamba-mphm-network
type: topic
---

# Multi-Prior Hierarchical Mamba Network

The Multi-Prior Hierarchical Mamba (MPHM) network is an architectural paradigm that systematically integrates heterogeneous prior knowledge within a hierarchical Mamba backbone to advance both image restoration and medical image analysis tasks. Across its instantiations in image deraining [2511.13113] and multi-phase contrast-enhanced CT tumor subtyping [2509.12777], MPHM is characterized by its fusion of complementary priors—semantic and structural—and its use of dual-path hierarchical modules (often leveraging the Mamba architecture) to capture multi-scale and domain-rich contextual information. MPHM achieves state-of-the-art performance in both domains, with rigorous ablation and evaluation protocols demonstrating the advantage of macro-micro prior integration and hierarchical dual-domain modeling.

## 1. Architectural Overview and Rationale

The defining architectural principle of MPHM is the fusion of distinct and complementary priors at multiple abstraction levels within a hierarchical network backbone.

**Deraining Context**: In single-image deraining, the objective is to separate rain streaks from scene content—requiring both macro-level semantic understanding (e.g., the concept “no rain on a car”) and micro-level structural discrimination (e.g., fine edges and textures). MPHM addresses these requirements using global semantic priors from the CLIP text encoder (prompted with “No rain”) and detailed structural priors from a frozen DINOv2 visual encoder. These priors are injected at each stage of a five-level U-shaped encoder-decoder network, whose backbone comprises Hierarchical Mamba Modules (HMM) operating in both spatial and frequency domains [2511.13113].

**CECT Tumor Subtyping Context**: In contrast-enhanced CT (CECT) analysis for tumor subtyping, MPHM as instantiated in CECT-Mamba processes spatial and temporal contrast patterns across phases (arterial, venous, delayed). The architecture integrates a 3D-CNN with a Dual-Hierarchical Contrast-enhanced-aware Mamba (DHCM) encoder, featuring both spatial and temporal tokenization and contrast-guided refinement [2509.12777].

The following table summarizes MPHM’s macro-architecture in its two major application domains:

| Instantiation              | Prior Types                        | Backbone Module        | Injection/Fusion Scheme          |
|----------------------------|------------------------------------|-----------------------|----------------------------------|
| Deraining [2511.13113]     | CLIP (text), DINOv2 (visual)       | HMM (dual-domain)     | Priors Fusion Injection (PFI)    |
| CECT-Mamba [2509.12777]    | Spatial/temporal context tokens    | DHCM (spatial/temp)   | Dual-hierarchical, SGR, MGF      |


## 2. Prior Fusion and Injection Mechanisms

A key innovation in MPHM is the progressive, level-wise injection of priors into the decoder or encoder streams via attention-based modules.

**Image Deraining – Priors Fusion Injection (PFI)**:
- At each decoder level \((l=1...5)\), both adapted visual prior \(P_v^l\) (from DINOv2) and textual prior \(P_t^l\) (from CLIP) are injected into the decoder feature \(F_b^l\) by sequential application of cross-attention (first with \(P_v^l\), then \(P_t^l\)), followed by self-attention and Gated Depth-wise Feedforward refinement.
- Mathematical formalization:
  $$
  F^{l}_{v\_in} = \mathrm{CrossAttn}(Q=P_v^l, K=F^l_b, V=F^l_b)
  $$
  $$
  F^{l}_{vt} = \mathrm{CrossAttn}(Q=P_t^l, K=F^l_{v\_in}, V=F^l_{v\_in}) + F^l_{v\_in}
  $$
  $$
  F_{\mathrm{PFI}}^l = \mathrm{GDFN}(\mathrm{SelfAttn}(F^{l}_{vt}) + F^{l}_{vt}) + \mathrm{SelfAttn}(F^{l}_{vt}) + F^{l}_{vt}
  $$
- This design contrasts with naive fusion (addition or concatenation), which empirically yields lower performance (e.g., PSNR 31.95 dB vs. 33.53 dB for hierarchical PFI on Rain200H [2511.13113]).

**CECT-Mamba – Dual-Hierarchical Tokenization**:
- At each encoder stage, feature maps are simultaneously processed via two tokenization streams:
  - **Spatial sampling**: patches extracted within each CECT phase, concatenated across phases.
  - **Temporal sampling**: per-voxel feature concatenation across the three phases.
- Temporal features with high inter-phase change are refined with a dedicated Mamba module (Similarity-Guided Refinement, SGR) before fusion with spatial branch outputs [2509.12777].

## 3. Hierarchical Mamba and Dual-Domain Feature Modeling

MPHM architectures exploit Mamba-based modules to model long-range dependencies and fine local features through specialized hierarchical designs.

**Hierarchical Mamba Module (HMM) – Deraining**:
- Operates with bifurcated spatial- and frequency-domain branches.
  - **Spatial branch**: Channel-split features processed via Visual Selective Spatial Mamba (VSSM) blocks and depth-wise convolutions, fused and refined through further Mamba and convolution layers.
  - **Frequency branch**: 2D FFT of input is passed through a lightweight Frequency-domain Feature Coupling Module (FFCM).
- The outputs of the two branches are concatenated, projected, and added residually:
  $$
  F_{\mathrm{out}} = F_{\mathrm{in}} + \mathrm{Conv}_{1\times1}(\mathrm{Concat}(F_{\mathrm{Spa}}, F_{\mathrm{Fre}}))
  $$
- Ablations demonstrate that both frequency-coupling (−2.50 dB PSNR if omitted) and depth-wise convolution are critical to restoration quality [2511.13113].

**Dual-Hierarchical Mamba – CECT-Mamba**:
- The DHCM block processes spatial-patch tokens and temporally sampled tokens (across CECT phases) via independent Mamba pathways.
- Temporal token refinement is channelled through SGR, focusing modeling capacity on regions of highest inter-phase variability, vital for discriminative tumor subtyping.
- Downsampling and multi-hierarchical feature extraction mirror U-Net and UNETR designs [2509.12777].

## 4. Training Objectives, Losses, and Hyperparameters

**Deraining [2511.13113]:**
- Total loss combines an pixel-space reconstruction loss and a frequency-domain contrastive regularization component:
  $$
  \mathcal{L}_{\rm total} = \mathcal{L}_{\rm rec} + \lambda \mathcal{L}_{\rm fcr}
  $$
  with
  $$
  \mathcal{L}_{\rm rec} = \frac{1}{N} \sum_{i=1}^N \|I_{\rm pred}^{(i)} - I_{\rm gt}^{(i)}\|_1
  $$
  $$
  \mathcal{L}_{\rm fcr} = \frac{1}{N} \sum_{i=1}^N \frac{ \|\mathcal{F}(I_{\rm gt}^{(i)}) - \mathcal{F}(I_{\rm pred}^{(i)})\|_1 }{ \|\mathcal{F}(I_{\rm rain*}^{(i)}) - \mathcal{F}(I_{\rm pred}^{(i)})\|_1 }
  $$
  where $\mathcal{F}$ denotes the Discrete Fourier Transform and $I_{\rm rain*}$ are random negatives.

- Training uses Adam optimizer, cosine-annealing from $1 \times 10^{-3}$ to $1 \times 10^{-5}$, patch crops sized $256 \times 256$, batch size 4, and backbone module depths $\{4, 6, 8, 6, 4\}$ [2511.13113].

**CECT-Mamba [2509.12777]:**
- Objective is standard cross-entropy over tumor class labels (PDAC, PNET); no contrastive or frequency-based regularization is applied.
- Optimization uses Adam, initial learning rate $1\times 10^{-5}$, cosine decay to $1\times 10^{-7}$ over 100 epochs, batch size 4, with heavy data augmentation (including random masking of up to 50% tokens in the spatial path).
- Preprocessing entails phase cropping ($128\times128\times32$), intensity normalization, and ROI localization via nnU-Net.

## 5. Empirical Performance and Ablation Analyses

MPHM consistently advances the empirical state of the art in its target domains.

**Deraining Results [2511.13113]:**
- Achieves PSNR 33.53 dB (+0.57 dB over TransMamba, +1.05 dB over FADformer) on Rain200H.
- Delivers perceptual improvements on real-world data: BRISQUE improves from 21.67 (NeRD-Rain) to 21.22, and NIQE from 3.84 to 3.79.
- Outperforms ablated configurations (e.g., single-prior or naive fusion), with both priors yielding PSNR 33.53 dB/SSIM 0.9475 versus 33.06 dB/0.9421 with no priors.

**CECT-Mamba Results [2509.12777]:**
- On an in-house 270-patient dataset, achieves 97.4% accuracy and 98.6% AUC for PDAC vs. PNET classification.
- Designs such as SGR and multi-granularity fusion are empirically validated for their contributions to final classification accuracy.

## 6. Data Flow, Pseudocode, and Pipeline Design

Both instantiations of MPHM provide explicit algorithmic schematics:

**Image Deraining Forward Pass [2511.13113]:**
```
function MPHM_forward(I_rain, text="No rain"):
    Pv_base = DINOv2_encoder(I_rain)  # frozen
    Pv_adapted = DINOv2_adapter(Pv_base)
    Pt_base = CLIP_text_encoder(text)  # frozen
    Pt_adapted = CLIP_adapter(Pt_base)

    F_enc[0] = I_rain
    for s in 1..5:
        F_enc[s] = downsample(HMM_s(F_enc[s-1]))

    F_dec[5] = F_enc[5]
    for s in 5..1:
        upl = upsample(F_dec[s])
        F_b  = concat(upl, F_enc[s-1])
        Pv_l = resize(Pv_adapted, size(F_b))
        Pt_l = resize(Pt_adapted, size(F_b))
        F_pfi = PFI(F_b, Pv_l, Pt_l)
        F_dec[s-1] = HMM_s−1(F_pfi)
    R = Conv_final(F_dec[0])
    I_pred = I_rain - R
    return I_pred
```

**CECT-Mamba Forward-Backward Iteration [2509.12777]:**
```
for each minibatch of B patients:
    # ROI localization and cropping
    ...
    # Initial 3D feature encoding + SCI
    ...
    # Dual-hierarchical Mamba encoder, spatial and temporal branches
    ...
    # Multi-granularity fusion & classification
    ...
    # Loss & backward
    ...
```
*Only data-provided pseudocode is shown; refer to [2511.13113] and [2509.12777] for detailed routines.*

## 7. Significance, Generalization, and Limitations

MPHM demonstrates clear advantages in multifaceted data integration and dual-domain feature modeling across vision and medical imaging settings. In deraining, its hierarchical prior fusion strategy delivers both higher fidelity restoration and superior perceptual metrics with moderate computational overhead (~10M parameters, ~62 GFLOPs) [2511.13113]. In CECT tumor subtyping, MPHM yields clinically significant gains in classification with explicit modeling of spatial-temporal context [2509.12777].

A plausible implication is that the MPHM design pattern—namely, systematic hierarchical fusion of heterogeneous priors and dual-domain contextualization—may readily generalize to other restoration, segmentation, or classification problems involving multi-modal or multi-phase inputs. Conversely, performance and computational cost may be sensitive to the design of prior adapters and the calibration of hierarchical fusion hyperparameters; ablation results in [2511.13113] confirm sensitivity to fusion strategy.

MPHM stands as an exemplar for the interplay between foundation model priors and structured hierarchical modeling in contemporary deep learning pipelines.

Source: https://www.emergentmind.com/topics/multi-prior-hierarchical-mamba-mphm-network