EndoCaver: Joint Endoscopic Deblurring & Segmentation

• EndoCaver is a lightweight transformer-based framework designed for joint deblurring and polyp segmentation in endoscopic images.
• It leverages a unidirectional-guided dual-decoder architecture with a MiT-B0 encoder, Global Attention Module, and Deblurring-Segmentation Aligner to enhance both visual restoration and segmentation.
• The method achieves high Dice scores (0.9221 clean, 0.8893 degraded) with low computational cost (7.81M parameters, 11.86 GMACs), making it suitable for on-device clinical deployment.

EndoCaver is a lightweight transformer-based framework for joint endoscopic image deblurring and polyp segmentation in colonoscopy images affected by lens fogging, motion blur, and specular glare/highlights. It is designed around the premise that segmentation becomes more reliable when the model first recovers cleaner visual structure, and that the two tasks should be learned together rather than through separate enhancement and segmentation pipelines. The method uses a unidirectional-guided dual-decoder architecture built on a MiT-B0 encoder from SegFormer, together with a Global Attention Module (GAM), a Deblurring-Segmentation Aligner (DSA), and a cosine-based loss scheduler termed LoCoS. On Kvasir-SEG, it reports 0.9221 Dice on clean data and 0.8893 Dice under severe image degradation, with a total size of 7.81M parameters and 11.86 GMACs, and the paper explicitly frames the model as suitable for on-device clinical deployment (Wu et al., 30 Jan 2026).

1. Problem setting and motivation

EndoCaver addresses a specific failure mode of endoscopic image analysis: real-world colonoscopy images are frequently degraded by fogging on the lens, motion blur, and specular highlights / glare. These corruptions lower contrast, obscure anatomy, and produce saturated bright regions and false edges. In polyp segmentation, the consequence is particularly acute because lesion boundary and texture can be lost, reducing the reliability of automated detection and delineation.

The method is motivated by several limitations in existing design patterns. The paper argues that plain encoder–decoder segmentation models are not robust enough under such degradations; dual-encoder or cascaded enhancement methods can be accurate but are often too heavy; two-stage enhancement-then-segmentation pipelines prevent efficient feature sharing; and large joint-learning models are difficult to deploy clinically. EndoCaver is therefore positioned as a compact alternative that preserves the benefits of restoration-aware segmentation without inheriting the computational cost of heavier multi-branch systems.

A common misconception is to treat image restoration as a purely preprocessing problem that should be completed before downstream analysis. EndoCaver rejects that separation. Its central claim is that restoration and segmentation should be trained jointly so that restored cues can directly shape task-specific mask prediction. This suggests that the architecture is not merely a pipeline compression exercise, but an attempt to reframe degraded endoscopic segmentation as a coupled inverse-and-discriminative problem.

2. Joint formulation and unidirectional guidance

The framework uses a single shared encoder followed by two task heads: a Deblurring decoder that predicts a restored image and a Segmentation decoder that predicts the polyp mask. The paper formulates the shared encoding as

$\{F_1, F_2, F_3, F_4\} = \text{Encoder}(I),$

where $I \in \mathbb{R}^{H \times W \times 3}$ is the input image and $F_i$ are multiscale features.

The outputs are

$$\hat{I} = \text{D-Decoder}(F_1,\dots,F_4), \qquad \hat{M} = \text{S-Decoder}(F_1,\dots,F_4,\hat{I}).$$

This formulation encodes the method’s defining asymmetry: the segmentation branch is unidirectionally guided by the deblurring branch. Restored visual cues are injected into segmentation, but not vice versa.

That asymmetry is central to the design logic. The deblurring branch is treated as a source of restoration priors, while the segmentation branch remains the terminal predictor of the clinically relevant output. The paper presents this as a computationally cheaper alternative to symmetric multi-branch interaction. A plausible implication is that the design aims to preserve task specialization while avoiding the overhead of bidirectional feature exchange.

3. Architectural components

The architecture has three main components: the MiT-B0 encoder from SegFormer for hierarchical feature extraction, the Global Attention Module (GAM) for lightweight cross-scale aggregation, and a unidirectional-guided dual-decoder connected through the Deblurring-Segmentation Aligner (DSA). The full model size is reported as 7.81M parameters and 11.86 GMACs.

GAM is introduced as a substitute for heavy skip-connection fusion. Rather than directly fusing all encoder scales in a U-Net-like manner, EndoCaver resizes encoder features to a common resolution, aggregates them, and applies attention. The paper states that all encoder features are normalized to a common spatial size, reduced through channel-wise averaging, and processed by multi-head attention,

$$\text{MHA}(Q,K,V) = \text{Softmax}\!\left(\frac{QK^\top}{\sqrt{d}}\right)V.$$

The resulting enhanced feature is fused back through bilinear upsampling and pointwise convolution, then combined with the original input by Hadamard product to form the enhanced multiscale representations $E_{1,2,3,4}$ used by the decoders. The stated intuition is that GAM collects global context across scales while suppressing corrupted local evidence at low computational cost.

DSA is the mechanism that transfers restoration knowledge from the deblurring decoder to segmentation through cross-attention. Given segmentation query feature $F_s$ and latent deblurring feature $F_d$, the first stage aligns segmentation with encoder-enhanced features: $F_s' = \text{CrossAttn}(F_s, F_{\text{enc}}).$ The second stage injects latent deblurring priors: $F_s'' = \text{CrossAttn}((F_s' + F_i), F_d).$ The paper describes this as unidirectional guidance: segmentation first incorporates the enhanced shared representation and is then refined using latent information learned by the deblurring branch. The refined feature is processed by a MobileViT block and upsampled to generate the final segmentation logits.

Taken together, these components instantiate the paper’s stated explanation for why the model works better: shared hierarchical encoding, global attention across scales, explicit restoration-to-segmentation transfer, and adaptive task balancing. In degraded endoscopic imagery, where boundaries and fine structures are easily lost, this arrangement is intended to recover useful cues and inject them where mask prediction is decided.

4. Optimisation, loss design, and training protocol

EndoCaver uses a joint loss with dynamic task weighting,

$I \in \mathbb{R}^{H \times W \times 3}$0

where

$I \in \mathbb{R}^{H \times W \times 3}$1

The segmentation weight follows cosine annealing,

$I \in \mathbb{R}^{H \times W \times 3}$2

This scheduler is named LoCoS. The paper interprets it as shifting optimization emphasis during training: early training places more focus on deblurring, whereas later training places stronger focus on segmentation. The claimed effect is more stable multi-task learning and improved generalization. Within the paper’s framing, LoCoS is not simply a weighting heuristic; it is part of the mechanism that coordinates the two tasks so that restoration priors mature before they are heavily relied on for segmentation.

The reported training setup is deliberately lightweight: PyTorch on a single NVIDIA A100 80G GPU, input resolution 224 × 224, batch size 16, Adam optimizer, learning rate $I \in \mathbb{R}^{H \times W \times 3}$3, and training with warmup plus cosine annealing. The training schedule distinguishes task phases: 3000 epochs for deblurring and 200 epochs for segmentation / EndoCaver. LoCoS balances the Dice and MSE losses.

5. Data regime, evaluation protocol, and quantitative performance

The primary dataset is Kvasir-SEG, comprising 900 images with an 80% training / 20% validation split. To assess generalization beyond the training domain, the paper also evaluates on CVC-ClinicDB and CVC-ColonDB. Real-world degradation is emulated through synthetic corruption including motion blur, defocus blur, specular highlights, and lens fogging. Segmentation quality is measured with Dice, IoU, and Recall; deblurring quality is measured with PSNR and SSIM. Higher is better for all reported metrics (Wu et al., 30 Jan 2026).

On Kvasir-SEG, EndoCaver reports 0.9221 Dice on clean data and 0.8893 Dice on degraded data, a drop of about 0.0328. The paper compares this with several baselines: UNet falls from 0.8699 to 0.8118, SegFormer-B5 from 0.9195 to 0.8623, CFFormer from 0.9127 to 0.8507, and H-Unets from 0.9011 to 0.8325. The paper accordingly states that EndoCaver has the best clean Dice, the best degraded Dice, and the smallest performance drop in that comparison.

On noisy Kvasir, the reported results are Dice 0.889, IoU 0.803, and Recall 0.872. For out-of-distribution transfer, the model achieves Dice 0.782 on Kvasir → ClinicDB and Dice 0.702 on Kvasir → ColonDB. The comparison set includes UNet, I2UNet-L, CTNet, NPDNet, CFFormer, SegFormer-B5, H-UNets, DRPViT, and cascaded pipelines.

Efficiency is a second major empirical claim. EndoCaver uses 7.81M parameters and 11.86 GMACs, compared with 99.56M for CFFormer, 81.97M for SegFormer-B5, and 31.04M for UNet. The paper highlights roughly 90% parameter reduction compared with some heavy baselines. The quantitative argument is therefore not only that EndoCaver is robust under corruption, but also that it compresses the restoration-segmentation design space into a clinically plausible computational envelope.

6. Ablation results, deployment implications, and relation to adjacent endoscopic vision

The ablation study isolates the contribution of each named component. The full model achieves Dice 0.8893, PSNR 23.10, 7.81M parameters, and 11.86 GMAC. Without LoCoS, performance drops to Dice 0.8822 and PSNR 22.35. Without LoCoS and DSA, the model reaches Dice 0.8801, PSNR 22.17, 7.28M parameters, and 9.91 GMAC. Without LoCoS, DSA, and GAM, the result is Dice 0.8756, PSNR 23.06, 7.08M parameters, and 4.36 GMAC. Without the deblurring branch, segmentation falls to Dice 0.8232, with 5.23M parameters and 1.68 GMAC.

The interpretation given in the paper is component-specific: LoCoS stabilizes training, DSA improves segmentation by transferring restoration priors, GAM contributes to multiscale global feature enhancement, and the deblurring branch is essential because removing it causes a large segmentation drop. Among these findings, the deblurring-branch ablation is the strongest evidence for the paper’s central thesis that restoration is not an optional auxiliary task but a structural prerequisite for robustness under fog, blur, and glare.

The paper explicitly presents EndoCaver as suitable for on-device clinical deployment and realistic for real-time endoscopy systems where compute and latency are constrained. That conclusion is tied to the coexistence of high reported accuracy, robustness under degradation, low parameter count, low GMACs, and lightweight transformer blocks. At the same time, the reported limitations are narrower than the deployment claim: evaluation is primarily on Kvasir-SEG with synthetic degradations and standard OOD datasets; deblurring still relies on the quality and diversity of training degradations; and the paper does not report deployment on actual embedded hardware or runtime latency on clinical devices. This suggests that the model’s clinical readiness is argued primarily from architecture-level efficiency and benchmark behavior rather than direct device-side validation.

Within the broader endoscopic computer vision ecosystem, EndoCaver should be distinguished from neighboring problem classes. EndoDAC focuses on self-supervised depth estimation from any endoscopic camera, using DV-LoRA, Convolutional Neck blocks, and camera-intrinsics estimation from monocular video; it is relevant to geometry and reconstruction rather than deblurring-guided 2D segmentation (Cui et al., 2024). PR-ENDO addresses physically based relightable Gaussian Splatting for endoscopy, targeting 3D reconstruction and novel-view synthesis under restricted camera rotations and strong view-dependent illumination (Kaleta et al., 2024). EndoCaver, by contrast, is a 2D joint restoration-segmentation model centered on polyp segmentation robustness under image degradation. A plausible implication is that these works are complementary rather than competing: EndoCaver targets degraded-frame semantic delineation, whereas EndoDAC and PR-ENDO target geometric inference and relightable 3D scene representation.