PolypSeg-GradCAM: Explainable Polyp Segmentation

• The paper presents PolypSeg-GradCAM as a novel framework that merges U-Net segmentation with Grad-CAM to produce pixel-wise polyp masks and visual explanations, enhancing model transparency.
• The method employs extensive preprocessing and data augmentation on the Kvasir-SEG dataset, ensuring robust performance across varied polyp sizes, shapes, and imaging conditions.
• Quantitative results, with a test IoU of 0.9257 and Dice score of 0.9612, demonstrate the framework's potential for reliable, explainable computer-aided colonoscopy.

PolypSeg-GradCAM is an explainable deep-learning framework for automatic polyp segmentation in colonoscopy images that combines a U-Net segmentation backbone with Gradient-weighted Class Activation Mapping (Grad-CAM) to produce both pixel-wise polyp masks and visual explanations of model focus (Asare et al., 17 Sep 2025). It is motivated by the clinical importance of gastrointestinal polyps as precursors of colorectal cancer according to the World Health Organization, and by the technical need to couple high segmentation accuracy with interpretable decision support in colonoscopy. The reported implementation is trained and evaluated on the Kvasir-SEG dataset of 1,000 annotated endoscopic images and is positioned as a framework for transparent, trustworthy AI-assisted colorectal cancer prevention (Asare et al., 17 Sep 2025).

1. Clinical and methodological setting

PolypSeg-GradCAM addresses automatic segmentation of gastrointestinal polyps in endoscopic imagery, a task that supports reliable detection, precise boundary delineation, lesion size estimation, and documentation and follow-up (Asare et al., 17 Sep 2025). The underlying clinical premise is that early detection and removal of polyps during colonoscopy greatly reduces colorectal cancer risk. The paper frames segmentation, rather than mere image-level classification, as central because clinically useful assistance depends on spatially resolved lesion localization and boundary fidelity.

The technical motivation arises from the difficulty of polyp segmentation in realistic colonoscopy scenes. The reported challenges include large variability in polyp size, shape, texture, and color, together with confounding structures and acquisition artifacts such as specular highlights, bubbles, stool, folds, and variable illumination (Asare et al., 17 Sep 2025). Manual delineation is described as time-consuming, labor-intensive, and affected by inter- and intra-observer variability, making it difficult to scale for large datasets or real-time assistance.

Within this setting, U-Net is selected because its encoder-decoder structure captures global semantics and local boundary details, skip connections preserve fine spatial information, and the architecture works well with relatively small medical-imaging datasets (Asare et al., 17 Sep 2025). The interpretability problem is then addressed by integrating Grad-CAM so that clinicians can inspect whether the model attends to polyp tissue rather than to artifacts or irrelevant anatomy. A common misconception is that high overlap metrics alone establish clinical trustworthiness; the framework explicitly rejects that assumption by treating visual explanation as a second design objective alongside segmentation accuracy.

2. Data basis and preprocessing pipeline

The framework is trained on Kvasir-SEG, described as a dataset of 1,000 colonoscopy images with corresponding binary masks, where mask value 0 denotes background and 255 denotes polyp (Asare et al., 17 Sep 2025). Image resolutions vary from 576×720 to 1920×1072, and the dataset includes strong variability in morphology, texture, color, illumination, specular highlights, and anatomical complexity. The paper characterizes this variability as making Kvasir-SEG a realistic benchmark for clinical deployment.

The initial split allocates 880 images to training and 120 to testing. After preprocessing, the training portion is further split into 792 images for training and 88 for validation, while the test set remains untouched for final evaluation, mirroring splits used in prior work on Kvasir-SEG (Asare et al., 17 Sep 2025).

Preprocessing follows four explicit stages. First, all images and masks are resized to 256×256 pixels as a trade-off between preserving anatomical detail and maintaining computational efficiency. Second, pixel intensities are normalized from $[0,255]$ to $[0,1]$, a step described as facilitating stable gradient updates and faster convergence. Third, each binary mask of shape $(H,W)$ is transformed to a one-hot representation $(H,W,2)$ with separate background and polyp channels, enabling per-pixel class probabilities. Fourth, training data alone are augmented by random horizontal flip $(p=0.5)$, random vertical flip $(p=0.5)$, random rotations $(\pm 15^\circ)$, and scaling transformations; after augmentation, images are resized back to $256\times256$ by bicubic interpolation to maintain shape consistency (Asare et al., 17 Sep 2025).

These design choices reflect a conventional medical-image segmentation pipeline, but they also delimit the scope of the reported results. Because all downstream evaluation is performed on resized images with binary masks, the framework is explicitly tuned to two-class segmentation under a fixed spatial resolution.

3. Network architecture and optimization objective

PolypSeg-GradCAM uses a standard U-Net implemented in PyTorch via Segmentation Models PyTorch (SMP) (Asare et al., 17 Sep 2025). The network follows the classical contracting-expanding design. In the encoder, each block contains two $3\times3$ convolutions followed by ReLU activations and a $2\times2$ max-pooling layer for downsampling. The textual architectural description reports feature depths of 64, 128, 256, and 512 channels along the contracting path, with the possibility of a 1024-channel bottleneck. The bottleneck is the deepest, lowest-resolution representation and is described as encoding global semantic information about polyp presence and context.

The decoder reverses this process through $[0,1]$0 transposed convolutions for upsampling, concatenation with encoder features through skip connections, and two further $[0,1]$1 convolutions with ReLU at each stage (Asare et al., 17 Sep 2025). Skip connections are emphasized as crucial for recovering high-resolution details lost during downsampling and for improving the detection of small or flat polyps. The output layer is a $[0,1]$2 convolution mapping features to two channels—background and polyp—followed by softmax to obtain per-pixel class probabilities. The description notes no explicit mention of batch normalization, dropout, or attention modules, and presents the implementation as close to the classical U-Net design.

Training uses soft Dice loss,

$[0,1]$3

where $[0,1]$4 is the predicted probability map, $[0,1]$5 is the ground-truth mask, and $[0,1]$6 is a small constant for numerical stability (Asare et al., 17 Sep 2025). The loss is chosen because it directly optimizes overlap and is suitable for class-imbalanced segmentation, where polyp pixels are often much fewer than background pixels.

Optimization uses Adam with an initial learning rate of $[0,1]$7 for up to 50 epochs, with early stopping triggered when validation metrics plateau (Asare et al., 17 Sep 2025). IoU and Dice are monitored on thresholded predictions for evaluation. This training configuration places the framework within a familiar supervised dense-prediction regime: architectural simplicity is preserved, and interpretability is introduced not by modifying the optimization target toward explanation-specific supervision, but by attaching a post hoc explanatory mechanism to the trained segmenter.

4. Grad-CAM integration and interpretability

The framework’s explainability component is Grad-CAM, used to visualize image regions most influential for the polyp class (Asare et al., 17 Sep 2025). For a target class $[0,1]$8 and feature maps $[0,1]$9 in a selected convolutional layer, the reported formulation computes channel-importance weights

$(H,W)$0

followed by the class activation map

$(H,W)$1

The ReLU retains only positive contributions, highlighting regions that support the class prediction.

In PolypSeg-GradCAM, Grad-CAM is applied to the last decoder layer of U-Net rather than to an encoder bottleneck or an image-level classification head (Asare et al., 17 Sep 2025). The rationale given is that this layer already integrates high-level semantic information with detailed spatial features and is therefore well aligned with the final segmentation output. The target class is the polyp class. The operational sequence consists of a forward pass to obtain segmentation probabilities, selection of a polyp-class score, backpropagation to the last decoder layer, computation of $(H,W)$2, formation of the Grad-CAM heatmap, upsampling to input-image size, and overlay on the original endoscopic frame (Asare et al., 17 Sep 2025).

The interpretive role of these heatmaps is explicitly diagnostic rather than merely illustrative. They are intended to reveal whether the network focuses on clinically relevant polyp tissue or instead drifts toward mucosal folds, bubbles, or specular highlights. This supports clinician-side validation of model behavior, identification of failure patterns, and human-in-the-loop review of borderline cases (Asare et al., 17 Sep 2025).

More broadly, explainability for semantic segmentation has been formalized by SEG-GRAD-CAM, which extends Grad-CAM from image-level logits to aggregated pixel-level class scores over a selected set of pixels $(H,W)$3 (Vinogradova et al., 2020). This situates PolypSeg-GradCAM within a segmentation-aware CAM lineage. However, the reported PolypSeg-GradCAM implementation is described specifically as U-Net segmentation plus Grad-CAM visualization at the last decoder layer, rather than as a full adoption of the localized pixel-selection formalism of SEG-GRAD-CAM (Asare et al., 17 Sep 2025).

5. Evaluation protocol, quantitative performance, and qualitative behavior

Evaluation uses Intersection-over-Union and Dice coefficient. The reported definitions are

$(H,W)$4

with Precision and Recall also monitored during training (Asare et al., 17 Sep 2025).

The reported numerical results are summarized below.

Setting	IoU	Dice / F-score
Training (selected later epochs)	0.9553 → 0.9729	0.9770 → 0.9862
Validation (representative range)	0.9189–0.9305	0.9571–0.9634
Test (120 held-out images)	0.9257	0.9612

On the held-out test set of 120 images, the framework achieves mean IoU = 0.9257 and mean F-score (Dice) = 0.9612 (Asare et al., 17 Sep 2025). Training curves are described as smooth and stable, with IoU surpassing 0.97 in later epochs and Dice exceeding 0.98. Validation performance remains near IoU 0.92–0.93 with Dice consistently above 0.95. The paper interprets the closeness of training, validation, and test scores as evidence of strong generalization within Kvasir-SEG.

The comparative baseline given is ResUNet on Kvasir-SEG from Jha et al. (2020), reported with Dice 0.7878 and IoU 0.7778 (Asare et al., 17 Sep 2025). Against that baseline, PolypSeg-GradCAM is described as substantially improved. The paper also notes that cross-dataset comparison with results such as U-Net on CVC-ClinicDB is not strictly meaningful, even though such results can suggest competitiveness (Asare et al., 17 Sep 2025).

Qualitative analysis is based on examples showing original colonoscopy images, ground-truth masks, predicted masks, and overlays. The model is reported to segment polyps across different sizes, shapes, textures, colors, and challenging illumination conditions, including both protruding and subtle flat lesions (Asare et al., 17 Sep 2025). Reported failure modes include slight under-segmentation of small or flat polyps, attention spillover onto adjacent mucosal folds, occasional attraction to strong reflections or bubbles, and the relative coarseness of Grad-CAM maps at fine boundaries. This is an important interpretive boundary: the heatmap explains regional focus, but it is not presented as a pixel-exact explanation of the final contour.

6. Clinical significance, limitations, and relation to later explainability work

The framework’s clinical relevance is framed around three linked claims: high-accuracy segmentation, explainability via Grad-CAM, and human-in-the-loop utility (Asare et al., 17 Sep 2025). IoU near 0.93 and Dice above 0.96 are described as strong enough for computer-aided detection, lesion size measurement, and documentation support. Grad-CAM overlays are presented as a way to confirm that decisions are guided by polyp regions and to expose mixed attention in difficult cases. In turn, this transparency is connected to endoscopist trust, regulatory acceptance under explainable-AI requirements, and iterative error analysis.

The reported limitations are equally explicit. Kvasir-SEG contains 1,000 images from a limited set of sources, so performance may degrade across centers, devices, or patient populations (Asare et al., 17 Sep 2025). Small, flat, or partially obscured polyps remain harder to segment. Grad-CAM heatmaps are regionally informative but relatively coarse and imperfect at the level of fine polyp edges. The model is also restricted to binary segmentation of polyp versus background, excluding ulcers, bleeding, inflammation, and other gastrointestinal findings. These constraints temper any interpretation of the framework as a complete GI analysis system.

The future directions suggested in the paper include larger and more diverse multi-center datasets, semi-supervised or few-shot learning, transformer-based designs such as TransUNet, Swin-UNet, and META-UNet, attention-augmented U-Net variants, and the combination of Grad-CAM with guided Grad-CAM, integrated gradients, or concept-based explanations (Asare et al., 17 Sep 2025). This suggests a trajectory in which explanation becomes finer-grained and more tightly coupled to model architecture.

Related work sharpens that implication. The C-Score metric, proposed for CAM-based explainability in medical image classification, quantifies intra-class explanation reproducibility through intensity-emphasised pairwise soft IoU across correctly classified instances (Elangovan et al., 9 Apr 2026). Although it is not applied in PolypSeg-GradCAM, it suggests a plausible evaluation axis for future explainable polyp-segmentation systems: not only whether a heatmap appears clinically plausible on a single image, but whether explanation behavior is stable across many cases of the same pathology. A separate line of polyp-segmentation research, exemplified by GRAFNet, does not explicitly compute classical Grad-CAM but instead builds interpretable attention and feedback into the segmentation architecture itself through modules such as the Guided Asymmetric Attention Module, the MultiScale Retinal Module, and the Guided Cortical Attention Feedback Module (Fofanah et al., 15 Feb 2026). Taken together, these developments indicate two complementary strategies for explainable polyp analysis: post hoc visualization of model focus, as in PolypSeg-GradCAM, and intrinsic architectural design for structured attention.