Reload-Mamba: Hierarchical Anti-Dilution State-Space Modeling for Multi-Class Semantic Segmentation

Abstract: Mamba-based state space models offer linear-time long-range modeling for high-resolution dense prediction, but sequential state-space propagation can attenuate boundary-sensitive and detail-sensitive responses that are critical in multi-class semantic segmentation. We propose Reload-Mamba, a semantic segmentation framework that addresses this propagation-induced response dilution through three segmentation-specific designs: (i) a boundary-supervised local detail prior that is explicitly trained with ground-truth boundary masks to identify regions requiring response restoration; (ii) a class-uncertainty-aware Reload Gate that incorporates per-pixel class entropy from a pre-reload auxiliary head as an additional gating signal, a formulation that is informative only under multi-class dense prediction; and (iii) a hierarchical multi-level Reload mechanism that applies anti-dilution refinement at three decoder levels and fuses the restored representations top-down. Built upon a ConvNeXt-Tiny encoder with a multi-scale decoder and four-directional Mamba scanning with pixel-wise directional attention, Reload-Mamba achieves 47.9% single-scale (48.9% multi-scale) mIoU on ADE20K and 83.2% single-scale mIoU on Cityscapes. With ResNet-101 + COCO pre-training under the standard DeepLab-style protocol, Reload-Mamba reaches 87.8% mIoU on PASCAL VOC 2012 val. Controlled ablations show that each of the three segmentation-specific designs contributes beyond a direct port of the prior anti-dilution architecture proposed for binarization, cumulatively improving over the direct-port baseline by +2.2 mIoU on ADE20K.

• The paper introduces a boundary-supervised local detail prior to precisely localize response dilution in semantic segmentation.
• It incorporates a class-uncertainty-aware Reload Gate that leverages per-pixel entropy to enhance ambiguous pixel restoration.
• The hierarchical multi-level refinement yields consistent mIoU gains across benchmarks like ADE20K, Cityscapes, and PASCAL VOC.

Hierarchical Anti-Dilution State-Space Modeling for Multi-Class Semantic Segmentation: An Analysis of Reload-Mamba

Introduction

Reload-Mamba presents a significant advance in state-space-model-driven semantic segmentation by targeting a core challenge of sequential state space propagation—response dilution, particularly at object boundaries and detail-sensitive regions. Standard Mamba-based approaches provide linear-time long-range dependency modeling but at the cost of attenuating discriminative features critical for pixel-precise multi-class segmentation. Reload-Mamba addresses this limitation through three distinct architectural innovations: a boundary-supervised local detail prior, a class-uncertainty-aware Reload Gate, and hierarchical multi-level anti-dilution refinement.

Motivation and Context

State space models (SSMs), especially Mamba and its vision adaptations, have surfaced as credible attention-free alternatives to Transformers for dense vision tasks due to their favorable scaling properties (Gu et al., 2023, Zhu et al., 2024, Huang et al., 2024, Chan et al., 7 Jun 2026). However, when directly ported to semantic segmentation, standard state-space propagation irreversibly mixes local and non-local features, weakening responses for small objects, thin structures, and boundaries—phenomena collectively described as propagation-induced response dilution. Prior work on anti-dilution gating in single-level, binary image binarization tasks (Chan et al., 7 Jun 2026) is inadequate for multi-class semantic segmentation, where such regions are numerous, ambiguous, and not localized by class probabilities alone.

Three critical shortcomings prevent direct transfer from binarization to segmentation: (1) boundary-sensitive regions are not inferable purely from conventional class probabilities; (2) semantic uncertainty (per-pixel entropy) is not leveraged, even though it is decisive in multi-class settings; and (3) information dilution propagates at multiple semantic scales, requiring coordinated multi-resolution restoration. Reload-Mamba systematically addresses each aspect via explicitly supervised priors, uncertainty-aware gating, and hierarchical multi-level integration.

Architecture and Methodology

The backbone network comprises ConvNeXt-Tiny, producing hierarchical features that are input to a standard multi-scale decoder. The Reload-Mamba module is applied at three decoder levels (D4, D3, D2), each incorporating three core mechanisms:

1. Boundary-Supervised Local Detail Prior: Each decoder level generates a prior map via explicit supervision using ground-truth boundary masks, replacing implicit priors in prior work. This mechanism ensures stable and spatially accurate localization of detail-sensitive pixels, especially those that cannot be inferred from class probabilities or feature activations alone.
2. Class-Uncertainty-Aware Reload Gate: A lightweight auxiliary head computes per-pixel class entropy (normalized), which, together with the original decoder feature and the Mamba-propagated contextual feature, feeds into a learned Reload Gate. In multi-class settings ($C > 2$), entropy directly highlights ambiguous pixels, coinciding with regions needing restoration—this is a discriminative signal absent from binary tasks.
3. Hierarchical Multi-Level Reload and Top-Down Fusion: Anti-dilution refinement is performed at three decoder levels, followed by progressive top-down fusion. This multi-scale approach acknowledges that dilution affects different semantic structures disparately (large regions at coarser scales, fine structures at finer scales), allowing for adaptive and resolution-specific restoration.

All modules are kept computationally efficient with careful width selection, and the pipeline maintains compatibility with various backbone architectures, extending beyond ConvNeXt (e.g., ResNet).

Empirical Evaluation

Quantitative Results

Reload-Mamba is evaluated on ADE20K, Cityscapes, and PASCAL VOC under standard protocols. Notable results include:

• ADE20K (150 classes): 47.9% single-scale mIoU with ConvNeXt-Tiny backbone (48.9% with multi-scale and flipping), outperforming the plain ConvNeXt-T (46.0%), matching or surpassing VMamba-T, and providing a +2.2 mIoU cumulative gain over single-level anti-dilution gating.
• Cityscapes (19 classes): 83.2% mIoU (single-scale), outperforming ConvNeXt-sized Transformer and Mamba baselines.
• PASCAL VOC (21 classes, ResNet-101 + COCO pretraining): 87.8% mIoU, matching DeepLabv3+ and EMANet under identical settings.

Ablation Studies

Empirical analysis confirms:

• Each segmentation-specific innovation (boundary-supervision, entropy-based gating, hierarchical refinement) contributes uniquely and additively. The class-uncertainty input yields the largest standalone gain.
• Hierarchical multi-level gating is superior to all single-level placements, regardless of decoder stage, and markedly better than late (post-upsampling) refinement.
• Directional state-space propagation (utilizing four scan directions) outperforms axis-aligned or single-direction scans, reinforcing the importance of directional context aggregation.
• Explicit boundary-supervised prior, as opposed to implicit estimation, consistently boosts mIoU and improves the spatial localization of restored details.
• The Reload-Mamba module is compatible with multiple encoder families, with the largest relative improvements seen in hierarchical, high-capacity backbones.

Qualitative Results

Predictions show superior boundary fidelity and sensitivity to small and thin semantic structures, confirming the theoretical advantages of explicitly restoring diluted responses.

Theoretical and Practical Implications

Reload-Mamba establishes that propagation-induced dilution is a fundamental architectural failure mode of sequential state-space models in high-resolution, multi-class dense prediction. Task-agnostic designs that do not incorporate explicit priors, uncertainty-aware gating, or hierarchical refinement are suboptimal in practice. The paper provides evidence that multi-resolution, task- and class-sensitive anti-dilution correction can close or exceed the gap with SOTA attention-based models, with lower inference cost and favorable scaling. The module’s design is inherently modular and is anticipated to combine synergistically with recent advances in backbone SSMs, e.g., VMamba (Zhu et al., 2024), Spatial-Mamba (Xiao et al., 2024), and SegMAN (Ma et al., 2024).

Future Directions

Future work should benchmark Reload-Mamba on broader task suites, including panoptic segmentation, instance segmentation, and depth estimation; explore content-adaptive and fully learnable scan path strategies to further improve directional context modeling; and integrate the anti-dilution mechanism into emerging SSM backbones to maximize joint gains in representation quality and efficiency. Architectural orthogonality to backbone design suggests that future SOTA segmentation pipelines are likely to incorporate modules specifically designed for anti-dilution correction in sequential context propagation.

Conclusion

Reload-Mamba demonstrates that high-fidelity semantic segmentation with SSMs is contingent on addressing task- and class-specific failure modes—most prominently, propagation-induced response dilution. Through boundary-supervision, class-uncertainty-aware gating, and hierarchical refinement, Reload-Mamba consistently improves mIoU across datasets and backbones over direct SSM transfers. These findings establish a blueprint for future architectural design of SSM-based dense predictors, underscoring the necessity for multi-level, uncertainty- and boundary-aware response restoration in complex semantic segmentation tasks.

---

Reference:

"Reload-Mamba: Hierarchical Anti-Dilution State-Space Modeling for Multi-Class Semantic Segmentation" (2606.17966)