ASGNet: Adaptive Spectrum Guidance Network for Automatic Polyp Segmentation

Abstract: Early identification and removal of polyps can reduce the risk of developing colorectal cancer. However, the diverse morphologies, complex backgrounds and often concealed nature of polyps make polyp segmentation in colonoscopy images highly challenging. Despite the promising performance of existing deep learning-based polyp segmentation methods, their perceptual capabilities remain biased toward local regions, mainly because of the strong spatial correlations between neighboring pixels in the spatial domain. This limitation makes it difficult to capture the complete polyp structures, ultimately leading to sub-optimal segmentation results. In this paper, we propose a novel adaptive spectrum guidance network, called ASGNet, which addresses the limitations of spatial perception by integrating spectral features with global attributes. Specifically, we first design a spectrum-guided non-local perception module that jointly aggregates local and global information, therefore enhancing the discriminability of polyp structures, and refining their boundaries. Moreover, we introduce a multi-source semantic extractor that integrates rich high-level semantic information to assist in the preliminary localization of polyps. Furthermore, we construct a dense cross-layer interaction decoder that effectively integrates diverse information from different layers and strengthens it to generate high-quality representations for accurate polyp segmentation. Extensive quantitative and qualitative results demonstrate the superiority of our ASGNet approach over 21 state-of-the-art methods across five widely-used polyp segmentation benchmarks. The code will be publicly available at: https://github.com/CSYSI/ASGNet.

• The paper introduces a spectrum-guided non-local perception module that fuses FFT-based spectral filtering with pixel-domain self-attention for enhanced global and local feature capture.
• It employs a multi-source semantic extractor combining atrous convolutions and global branches to robustly localize polyps even with ambiguous boundaries.
• The dense cross-layer interaction decoder integrates multi-level features with explicit edge enhancement to deliver precise segmentation masks, validated on five polyp datasets.

ASGNet: Adaptive Spectrum Guidance Network for Automatic Polyp Segmentation

Introduction and Motivation

Automatic and accurate polyp segmentation in colonoscopy is critical for early colorectal cancer (CRC) screening. Traditional approaches suffer from limited local spatial perception and struggle to capture the global morphological diversity and the ambiguous boundaries of polyps in complex backgrounds. Existing CNN- and Transformer-based segmentation methods partially address these issues via feature aggregation, self-attention, and multiscale analysis, but often remain biased toward local regions and lack sufficient holistic discrimination. The "ASGNet: Adaptive Spectrum Guidance Network for Automatic Polyp Segmentation" (2604.14755) introduces a framework that systematically integrates spectral domain information to optimize both local and global feature perception, proposing a modular architecture combining spectrum-guided non-local perception, multi-source semantic extraction, and dense cross-layer decoding.

Main Contributions

The key innovations of ASGNet are as follows:

• Spectrum-Guided Non-Local Perception (SNP) Module: Combines pixel-domain self-attention with an adaptive spectrum filter (ASF) based on Fast Fourier Transform (FFT) to encode both global and local semantic contexts.
• Multi-Source Semantic Extractor (MSE): Employs multiple atrous convolutional branches and two global branches (GAP and ASF) to aggregate diverse high-level semantic representations for robust coarse localization.
• Dense Cross-Layer Interaction (DCI) Decoder: Aggregates and reinforces multi-level features and semantic cues across stages with explicit edge and object enhancement mechanisms to deliver fine-grained segmentation masks with sharp structural details.

Methodology

Spectrum-Guided Non-Local Perception

The SNP module is designed to address the limited receptive field and local bias of pixel-wise modeling. By applying the FFT to feature maps, spectrum features encompassing the full frequency range are retrieved without splitting into high and low frequency bands, thus avoiding the information bottleneck observed in previous spectral guidance methods. An adaptive attention mechanism in the spectrum domain then enhances salient bands and suppresses redundancy. Spatial (self-attention) and frequency domain signals are fused, and local context is further sharpened with convolutional blocks for precise boundary refinement. The module is integrated throughout the encoder to maximize the propagation of global semantic coherence.

Figure 2: Qualitative comparison of segmentation results using individual LEB, SAM, and ASF submodules demonstrates the impact of each in the SNP mechanism.

Multi-Source Semantic Extraction

The MSE block facilitates robust polyp localization under ambiguous global context. Six dense local branches employ atrous convolutions with increasing dilation rates to extract features with diverse receptive fields. Two global branches (GAP and ASF) supply image-level semantic abstraction and spectrum features. After feature fusion, the MSE yields a coarse location prior which effectively guides subsequent fine-grained decoding.

Dense Cross-Layer Interaction Decoding

The DCI decoder integrates the outputs of SNP and MSE modules across all stages. It fuses hierarchical representations using channel-wise dense connections, and separately enhances edge information (multi-scale spatial attention and explicit gradient cueing) and object features (ASF enhancement and reverse attention). Prediction at each decoding stage is supervised both at the mask and edge levels, and loss aggregation includes weighted BCE, weighted IoU, and Dice terms.

Experimental Results

ASGNet is benchmarked on five major public polyp datasets: CVC-300, CVC-ColonDB, ETIS-Larib, Kvasir, and CVC-ClinicDB. It is compared against 21 state-of-the-art methods across both CNN and Transformer paradigms.

Empirical results indicate:

• Consistent gains on all metrics: On CVC-300, ASGNet achieves Dice scores of 0.912 (ResNet-50) and 0.909 (PVTv2B3), outperforming contemporaries such as UHANet, PolypPVT, PPNet, and LSSNet.
• Robust generalization: The performance advantage extends to more difficult datasets with small or ambiguous polyps (ETIS-Larib, CVC-ColonDB).
• Efficiency: ASGNet maintains a competitive number of parameters, FLOPs, and inference speed while surpassing models with larger backbone complexity.

Ablation studies reveal each ASGNet module (SNP, MSE, DCI) delivers non-trivial absolute gains, and spectral integration is a critical enabler for global-context-aware segmentation.

Implications and Future Directions

The demonstrated effectiveness of spectral guidance, especially when adaptively weighted and embedded throughout both encoding and decoding, signifies the importance of leveraging frequency domain representations in medical image analysis, and potentially other domains characterized by weak or ambiguous spatial cues.

Key theoretical takeaways:

• Frequency information, when properly modeled and adaptively filtered, supplies an orthogonal global context channel that is complementary to spatial attention even in advanced Transformer-based frameworks.
• Dense multi-source aggregation and spectrum-enhanced decoding address the persistent challenges of boundary ambiguity and small object segmentation.

Practically, this yields a tool for robust, accurate clinical polyp detection; deployment as a physician-assist system is facilitated by its reasonable inference cost.

Looking forward, future research avenues include tailored spectrum augmentation for extremely small polyp detection, exploration of spectrum-guided approaches in other challenging medical segmentation tasks, and scalable transfer and adaptation to other image and video domains with similar structural properties.

Conclusion

ASGNet exemplifies the integration of spectral domain analysis with advanced semantic aggregation and decoding for automatic polyp segmentation. Its modular spectrum-guided design notably advances both quantitative accuracy and qualitative mask quality, especially in challenging clinical cases with low contrast or complex background structure. The architectural design and empirical evidence from this work provide a robust foundation and set new benchmarks for the broader class of medical and biological image segmentation under real-world imaging constraints (2604.14755).