---
title: 'EndoCaver: Joint Endoscopic Deblurring & Segmentation'
url: https://www.emergentmind.com/topics/endocaver
type: topic
---

# EndoCaver: Joint Endoscopic Deblurring & Segmentation

EndoCaver is a lightweight transformer-based framework for **joint endoscopic image deblurring and polyp segmentation** in colonoscopy images affected by **lens fogging, motion blur, and specular glare/highlights**. It is designed around the premise that segmentation becomes more reliable when the model first recovers cleaner visual structure, and that the two tasks should be learned together rather than through separate enhancement and segmentation pipelines. The method uses a **unidirectional-guided dual-decoder architecture** built on a **MiT-B0 encoder** from SegFormer, together with a **Global Attention Module (GAM)**, a **Deblurring-Segmentation Aligner (DSA)**, and a cosine-based loss scheduler termed **LoCoS**. On **Kvasir-SEG**, it reports **0.9221 Dice** on clean data and **0.8893 Dice** under severe image degradation, with a total size of **7.81M parameters** and **11.86 GMACs**, and the paper explicitly frames the model as suitable for **on-device clinical deployment** [2601.22537].

## 1. Problem setting and motivation

EndoCaver addresses a specific failure mode of endoscopic image analysis: real-world colonoscopy images are frequently degraded by **fogging** on the lens, **motion blur**, and **specular highlights / glare**. These corruptions lower contrast, obscure anatomy, and produce saturated bright regions and false edges. In **polyp segmentation**, the consequence is particularly acute because lesion boundary and texture can be lost, reducing the reliability of automated detection and delineation.

The method is motivated by several limitations in existing design patterns. The paper argues that plain encoder–decoder segmentation models are not robust enough under such degradations; dual-encoder or cascaded enhancement methods can be accurate but are often too heavy; two-stage enhancement-then-segmentation pipelines prevent efficient feature sharing; and large joint-learning models are difficult to deploy clinically. EndoCaver is therefore positioned as a compact alternative that preserves the benefits of restoration-aware segmentation without inheriting the computational cost of heavier multi-branch systems.

A common misconception is to treat image restoration as a purely preprocessing problem that should be completed before downstream analysis. EndoCaver rejects that separation. Its central claim is that restoration and segmentation should be trained jointly so that restored cues can directly shape task-specific mask prediction. This suggests that the architecture is not merely a pipeline compression exercise, but an attempt to reframe degraded endoscopic segmentation as a coupled inverse-and-discriminative problem.

## 2. Joint formulation and unidirectional guidance

The framework uses a single shared encoder followed by two task heads: a **Deblurring decoder** that predicts a restored image and a **Segmentation decoder** that predicts the polyp mask. The paper formulates the shared encoding as
\[
\{F_1, F_2, F_3, F_4\} = \text{Encoder}(I),
\]
where \(I \in \mathbb{R}^{H \times W \times 3}\) is the input image and \(F_i\) are multiscale features.

The outputs are
\[
\hat{I} = \text{D-Decoder}(F_1,\dots,F_4), \qquad
\hat{M} = \text{S-Decoder}(F_1,\dots,F_4,\hat{I}).
\]
This formulation encodes the method’s defining asymmetry: the segmentation branch is **unidirectionally guided by the deblurring branch**. Restored visual cues are injected into segmentation, but not vice versa.

That asymmetry is central to the design logic. The deblurring branch is treated as a source of restoration priors, while the segmentation branch remains the terminal predictor of the clinically relevant output. The paper presents this as a computationally cheaper alternative to symmetric multi-branch interaction. A plausible implication is that the design aims to preserve task specialization while avoiding the overhead of bidirectional feature exchange.

## 3. Architectural components

The architecture has three main components: the **MiT-B0 encoder** from SegFormer for hierarchical feature extraction, the **Global Attention Module (GAM)** for lightweight cross-scale aggregation, and a **unidirectional-guided dual-decoder** connected through the **Deblurring-Segmentation Aligner (DSA)**. The full model size is reported as **7.81M parameters** and **11.86 GMACs**.

GAM is introduced as a substitute for heavy skip-connection fusion. Rather than directly fusing all encoder scales in a U-Net-like manner, EndoCaver resizes encoder features to a common resolution, aggregates them, and applies attention. The paper states that all encoder features are normalized to a common spatial size, reduced through channel-wise averaging, and processed by multi-head attention,
\[
\text{MHA}(Q,K,V) = \text{Softmax}\!\left(\frac{QK^\top}{\sqrt{d}}\right)V.
\]
The resulting enhanced feature is fused back through **bilinear upsampling** and **pointwise convolution**, then combined with the original input by **Hadamard product** to form the enhanced multiscale representations \(E_{1,2,3,4}\) used by the decoders. The stated intuition is that GAM collects **global context across scales** while suppressing corrupted local evidence at low computational cost.

DSA is the mechanism that transfers restoration knowledge from the deblurring decoder to segmentation through **cross-attention**. Given segmentation query feature \(F_s\) and latent deblurring feature \(F_d\), the first stage aligns segmentation with encoder-enhanced features:
\[
F_s' = \text{CrossAttn}(F_s, F_{\text{enc}}).
\]
The second stage injects latent deblurring priors:
\[
F_s'' = \text{CrossAttn}((F_s' + F_i), F_d).
\]
The paper describes this as **unidirectional guidance**: segmentation first incorporates the enhanced shared representation and is then refined using latent information learned by the deblurring branch. The refined feature is processed by a **MobileViT block** and upsampled to generate the final segmentation logits.

Taken together, these components instantiate the paper’s stated explanation for why the model works better: **shared hierarchical encoding**, **global attention across scales**, **explicit restoration-to-segmentation transfer**, and **adaptive task balancing**. In degraded endoscopic imagery, where boundaries and fine structures are easily lost, this arrangement is intended to recover useful cues and inject them where mask prediction is decided.

## 4. Optimisation, loss design, and training protocol

EndoCaver uses a **joint loss** with dynamic task weighting,
\[
\mathcal{L}(t) = \big(1-w_{\text{seg}}(t)\big)\mathcal{L}_{\text{deb}} + w_{\text{seg}}(t)\mathcal{L}_{\text{seg}},
\]
where
\[
\mathcal{L}_{\text{deb}} = \| I - \hat{I} \|_2^2, \qquad
\mathcal{L}_{\text{seg}} = 1 - \text{Dice}(M,\hat{M}).
\]
The segmentation weight follows cosine annealing,
\[
w_{\text{seg}}(t) = w_{\min} + \tfrac{1}{2}(1-w_{\min})(1+\cos(\pi t / T)).
\]

This scheduler is named **LoCoS**. The paper interprets it as shifting optimization emphasis during training: **early training** places more focus on deblurring, whereas **later training** places stronger focus on segmentation. The claimed effect is more stable multi-task learning and improved generalization. Within the paper’s framing, LoCoS is not simply a weighting heuristic; it is part of the mechanism that coordinates the two tasks so that restoration priors mature before they are heavily relied on for segmentation.

The reported training setup is deliberately lightweight: **PyTorch** on a **single NVIDIA A100 80G GPU**, input resolution **224 × 224**, batch size **16**, **Adam** optimizer, learning rate **\(1 \times 10^{-4}\)**, and training with **warmup** plus **cosine annealing**. The training schedule distinguishes task phases: **3000 epochs** for deblurring and **200 epochs** for segmentation / EndoCaver. LoCoS balances the **Dice** and **MSE** losses.

## 5. Data regime, evaluation protocol, and quantitative performance

The primary dataset is **Kvasir-SEG**, comprising **900 images** with an **80% training / 20% validation** split. To assess generalization beyond the training domain, the paper also evaluates on **CVC-ClinicDB** and **CVC-ColonDB**. Real-world degradation is emulated through synthetic corruption including **motion blur**, **defocus blur**, **specular highlights**, and **lens fogging**. Segmentation quality is measured with **Dice**, **IoU**, and **Recall**; deblurring quality is measured with **PSNR** and **SSIM**. Higher is better for all reported metrics [2601.22537].

On Kvasir-SEG, EndoCaver reports **0.9221 Dice** on clean data and **0.8893 Dice** on degraded data, a drop of about **0.0328**. The paper compares this with several baselines: **UNet** falls from **0.8699** to **0.8118**, **SegFormer-B5** from **0.9195** to **0.8623**, **CFFormer** from **0.9127** to **0.8507**, and **H-Unets** from **0.9011** to **0.8325**. The paper accordingly states that EndoCaver has the **best clean Dice**, the **best degraded Dice**, and the **smallest performance drop** in that comparison.

On noisy Kvasir, the reported results are **Dice 0.889**, **IoU 0.803**, and **Recall 0.872**. For out-of-distribution transfer, the model achieves **Dice 0.782** on **Kvasir → ClinicDB** and **Dice 0.702** on **Kvasir → ColonDB**. The comparison set includes **UNet**, **I2UNet-L**, **CTNet**, **NPDNet**, **CFFormer**, **SegFormer-B5**, **H-UNets**, **DRPViT**, and **cascaded pipelines**.

Efficiency is a second major empirical claim. EndoCaver uses **7.81M parameters** and **11.86 GMACs**, compared with **99.56M** for **CFFormer**, **81.97M** for **SegFormer-B5**, and **31.04M** for **UNet**. The paper highlights roughly **90% parameter reduction** compared with some heavy baselines. The quantitative argument is therefore not only that EndoCaver is robust under corruption, but also that it compresses the restoration-segmentation design space into a clinically plausible computational envelope.

## 6. Ablation results, deployment implications, and relation to adjacent endoscopic vision

The ablation study isolates the contribution of each named component. The **full model** achieves **Dice 0.8893**, **PSNR 23.10**, **7.81M parameters**, and **11.86 GMAC**. **Without LoCoS**, performance drops to **Dice 0.8822** and **PSNR 22.35**. **Without LoCoS and DSA**, the model reaches **Dice 0.8801**, **PSNR 22.17**, **7.28M parameters**, and **9.91 GMAC**. **Without LoCoS, DSA, and GAM**, the result is **Dice 0.8756**, **PSNR 23.06**, **7.08M parameters**, and **4.36 GMAC**. **Without the deblurring branch**, segmentation falls to **Dice 0.8232**, with **5.23M parameters** and **1.68 GMAC**.

The interpretation given in the paper is component-specific: **LoCoS stabilizes training**, **DSA improves segmentation by transferring restoration priors**, **GAM contributes to multiscale global feature enhancement**, and the **deblurring branch is essential** because removing it causes a large segmentation drop. Among these findings, the deblurring-branch ablation is the strongest evidence for the paper’s central thesis that restoration is not an optional auxiliary task but a structural prerequisite for robustness under fog, blur, and glare.

The paper explicitly presents EndoCaver as suitable for **on-device clinical deployment** and realistic for **real-time endoscopy systems** where compute and latency are constrained. That conclusion is tied to the coexistence of high reported accuracy, robustness under degradation, low parameter count, low GMACs, and lightweight transformer blocks. At the same time, the reported limitations are narrower than the deployment claim: evaluation is primarily on **Kvasir-SEG** with synthetic degradations and standard OOD datasets; deblurring still relies on the quality and diversity of training degradations; and the paper does not report deployment on actual embedded hardware or runtime latency on clinical devices. This suggests that the model’s clinical readiness is argued primarily from architecture-level efficiency and benchmark behavior rather than direct device-side validation.

Within the broader endoscopic computer vision ecosystem, EndoCaver should be distinguished from neighboring problem classes. **EndoDAC** focuses on **self-supervised depth estimation from any endoscopic camera**, using **DV-LoRA**, **Convolutional Neck blocks**, and camera-intrinsics estimation from monocular video; it is relevant to geometry and reconstruction rather than deblurring-guided 2D segmentation [2405.08672]. **PR-ENDO** addresses **physically based relightable Gaussian Splatting for endoscopy**, targeting **3D reconstruction** and **novel-view synthesis** under restricted camera rotations and strong view-dependent illumination [2411.12510]. EndoCaver, by contrast, is a 2D joint restoration-segmentation model centered on polyp segmentation robustness under image degradation. A plausible implication is that these works are complementary rather than competing: EndoCaver targets degraded-frame semantic delineation, whereas EndoDAC and PR-ENDO target geometric inference and relightable 3D scene representation.

Source: https://www.emergentmind.com/topics/endocaver