AVPDN: Adaptive Video Polyp Detection

• The paper introduces AVPDN, a framework that enhances detection by integrating adaptive feature interaction (AFIA) and scale-aware context integration (SACI) into a RT-DETR baseline.
• AVPDN addresses video-specific challenges such as motion blur, scale variation, and transient artifacts in colonoscopy frames.
• Comprehensive evaluations on LDPolypVideo and CVC-ClinicVideoDB show that AVPDN outperforms standard RT-DETR with significant gains in average precision and F1 score.

Searching arXiv for AVPDN and closely related video polyp detection papers to ground the article in current literature. Adaptive Video Polyp Detection Network (AVPDN) is a video-based polyp detection framework for colonoscopy that is proposed to address motion-induced noise and scale variation in dynamic endoscopic video. In the paper "AVPDN: Learning Motion-Robust and Scale-Adaptive Representations for Video-Based Polyp Detection," AVPDN is built on RT-DETR and augments its feature extraction stage with two modules: Adaptive Feature Interaction and Augmentation (AFIA) and Scale-Aware Context Integration (SACI). The method is framed as a real-time detector for colonoscopy video frames, with the stated goal of learning motion-robust and scale-adaptive representations rather than performing explicit temporal sequence modeling (Chen et al., 5 Aug 2025).

1. Concept and problem setting

AVPDN is defined for video-based polyp detection in colonoscopy, where the input is described as the current frame of a colonoscopy video clip and the output is a standard object detection prediction consisting of class and bounding box for polyp instances (Chen et al., 5 Aug 2025). The motivating claim is that dynamic colonoscopy videos provide more comprehensive visual information than static images, but also introduce substantial background noise because the endoscope moves rapidly and the scene can undergo motion blur, background distortion, and apparent structural “fracture” of tissue patterns such as folds and vasculature (Chen et al., 5 Aug 2025).

The framework is positioned against several domain-specific sources of difficulty. The paper emphasizes motion blur, transient artifacts such as specular highlights, air bubbles, and fluid interference, and severe scale variation caused by changing camera-to-tissue distance. It also states that small polyps may resemble surrounding mucosa and can behave like partially occluded objects in the detector’s feature space, increasing both false positives and false negatives (Chen et al., 5 Aug 2025). This suggests that AVPDN treats colonoscopy video not primarily as a long-horizon temporal reasoning problem, but as a setting in which frame-wise detection must be made robust to video-induced corruption.

This design choice distinguishes AVPDN from prior video polyp detectors that explicitly aggregate adjacent frames. For example, Ivy-Net formulates image-to-video domain adaptation through modified mixup and temporal coherence regularization (Zhan et al., 2020), STFT performs proposal-guided deformable alignment and channel-aware temporal aggregation across multiple frames (Wu et al., 2021), and YONA uses one adjacent reference frame with foreground temporal alignment and background dynamic alignment (Jiang et al., 2023). By contrast, AVPDN emphasizes representation design inside a real-time detector rather than explicit temporal fusion (Chen et al., 5 Aug 2025).

2. Architectural organization

AVPDN is built on RT-DETR, with a ResNet-50 backbone, a series of Adaptive Feature Enhancement (AFE) blocks, and the Transformer encoder-decoder inherited from RT-DETR, including deformable attention in the decoder and auxiliary prediction heads (Chen et al., 5 Aug 2025). An input frame is denoted by

$I \in \mathbb{R}^{H \times W \times C},$

and the backbone produces a low-level representation

$F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$

where $s$ is the backbone downsampling factor, typically 32, and $C'$ is the channel dimension (Chen et al., 5 Aug 2025).

The AFE block is the main representational unit. Each block applies Layer Normalization, then AFIA, then another Layer Normalization, then SACI (Chen et al., 5 Aug 2025). The resulting enhanced multi-scale features are passed to the RT-DETR transformer decoder, which predicts bounding boxes and class labels. The paper does not specify the exact number of AFE blocks or the exact pyramid levels at which they are inserted, and it does not provide exact tensor shapes for each branch beyond generic $H \times W \times C$ notation (Chen et al., 5 Aug 2025).

This architecture is best understood as a domain-specialized RT-DETR variant. The backbone and decoder remain detector-standard, while the feature enhancement stage is modified to better accommodate colonoscopy video artifacts. A plausible implication is that AVPDN aims to preserve the throughput advantages of RT-DETR while improving robustness at the representation level.

3. AFIA and motion-robust feature interaction

The Adaptive Feature Interaction and Augmentation module is the more distinctive of AVPDN’s two additions. AFIA has a triple-branch architecture consisting of a channel shuffle branch, a dense self-attention branch, and a sparse self-attention branch (Chen et al., 5 Aug 2025). The paper motivates this design by arguing that ordinary Vision Transformer-style attention is not ideal in colonoscopy videos because motion blur, specular reflections, and repetitive background texture produce redundant or misleading interactions (Chen et al., 5 Aug 2025).

Given a normalized feature map

$X \in \mathbb{R}^{H \times W \times C},$

the channel shuffle branch is written as

$$\begin{aligned} F_{cs} &= SConv_{3\times3}({Conv}_{1\times1}({CS}(X)), \end{aligned}$$

where $F_{cs}$ is the branch output, $SConv_{3\times3}$ denotes stacked $3\times3$ convolutions, $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$0 is a $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$1 convolution, and $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$2 denotes channel shuffle (Chen et al., 5 Aug 2025). The paper notes that this equation is typographically incomplete, and it does not provide exact implementation details of the shuffle groups (Chen et al., 5 Aug 2025).

In the dense self-attention branch, $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$3, $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$4, and $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$5 are generated from $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$6 using $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$7 convolutions followed by $F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$8 depthwise convolutions. Dense self-attention is defined as

$F_0 \in \mathbb{R}^{H/s \times W/s \times C'},$9

where $s$0 is the feature dimension used for scaling and $s$1 is a learnable relative positional bias (Chen et al., 5 Aug 2025). This branch is intended to preserve global context.

The sparse self-attention branch replaces softmax with ReLU:

$s$2

so that low or negative query-key similarity pairs contribute nothing (Chen et al., 5 Aug 2025). The paper explicitly states that this branch mitigates the influence of low query-key similarity in feature aggregation, which is presented as useful under motion blur and specular reflection (Chen et al., 5 Aug 2025).

The dense and sparse branches are fused with learned weights:

$s$3

and

$s$4

where $s$5 are learnable parameters and $s$6 is the fused attention output (Chen et al., 5 Aug 2025). The final AFIA output is

$s$7

The paper interprets this as combining local and channel-mixed features with globally contextualized features (Chen et al., 5 Aug 2025).

The ablation study on LDPolypVideo attributes the largest gain to this dual-attention design. Starting from RT-DETR at AP 94.2, Precision 94.4, Recall 92.3, and F1 93.3, adding only DSA gives AP 95.5 and F1 94.9, adding only SSA gives AP 95.2 and F1 94.7, and combining DSA + SSA gives AP 96.3 and F1 95.5. Adding channel shuffle on top of DSA + SSA yields AP 96.6 and F1 95.6 (Chen et al., 5 Aug 2025). This suggests that the interaction between dense context retention and sparse suppression is central to AVPDN’s motion robustness.

4. SACI and scale-aware context integration

The Scale-Aware Context Integration module is designed to improve multi-scale context modeling and denoising (Chen et al., 5 Aug 2025). The paper motivates SACI by arguing that the standard transformer FFN behaves as a single-scale feature aggregator with limited contextual range. SACI replaces this with a dual-branch unit built on dilated and stacked convolutions (Chen et al., 5 Aug 2025).

Given input

$s$8

the upper branch uses two $s$9 dilated convolutions with different dilation rates:

$C'$0

where $C'$1 and $C'$2 are dilated convolutions with rates $C'$3 and $C'$4 (Chen et al., 5 Aug 2025). In experiments, the paper sets

$C'$5

The lower branch is defined as

$C'$6

where $C'$7 denotes stacked $C'$8 convolutions (Chen et al., 5 Aug 2025). The final SACI output is

$C'$9

According to the paper, the multiplication acts like a gating mechanism in which broader contextual responses are amplified where local evidence is strong (Chen et al., 5 Aug 2025).

SACI alone improves the RT-DETR baseline on LDPolypVideo to AP 95.0 and F1 94.7, and the full model with AFIA + SACI reaches AP 96.6, Precision 96.8, Recall 95.0, and F1 95.8 (Chen et al., 5 Aug 2025). The paper states that SACI improves performance more robustly on larger polyps, while AFIA and SACI together help smaller ones (Chen et al., 5 Aug 2025). This suggests that AVPDN’s scale adaptivity is achieved by embedding candidate regions in broader context rather than by explicit multi-frame size tracking.

5. Training, datasets, and reported performance

The paper explicitly defines its localization losses. With predicted box $H \times W \times C$0, ground-truth box $H \times W \times C$1, and smallest enclosing box $H \times W \times C$2, generalized IoU is given as

$H \times W \times C$3

with

$H \times W \times C$4

The $H \times W \times C$5 term is

$H \times W \times C$6

The total loss is written as

$H \times W \times C$7

The paper notes auxiliary prediction heads for localization and classification, but it does not provide a classification loss equation or assignment details, even though the architecture is built on RT-DETR (Chen et al., 5 Aug 2025). This is an important limitation of the published specification.

AVPDN is evaluated on two public video polyp detection datasets. LDPolypVideo contains 160 endoscopic videos and 40,266 image frames, with 33,884 frames containing at least one polyp and 200 annotated polyps. CVC-ClinicVideoDB contains 40 endoscopic videos and 17,000 frames, with 6,949 frames annotated for polyps. The paper states that all methods use a 7:2:1 split for training, test, and validation (Chen et al., 5 Aug 2025). All images are normalized and resized to $H \times W \times C$8, training runs for 60 epochs with batch size 32, optimizer AdamW, learning rate 0.001, weight decay 0.0005, and ImageNet-pretrained ResNet-50 initialization (Chen et al., 5 Aug 2025).

On LDPolypVideo, Table 1 reports for AVPDN: AP 96.6, Precision 96.8, Recall 95.0, F1 95.8, and FPS 53.2. The direct baseline RT-DETR reports AP 94.2, Precision 94.4, Recall 92.3, F1 93.3, and FPS 52.1 (Chen et al., 5 Aug 2025). On CVC-VideoClinicDB, AVPDN reports AP 95.7, Precision 95.9, Recall 94.9, F1 95.3, and FPS 53.2, compared with RT-DETR at AP 93.1, Precision 94.1, Recall 93.9, F1 93.9, and FPS 52.1 (Chen et al., 5 Aug 2025). The paper interprets these margins as evidence of effectiveness and generalization under variations in illumination, camera angle, and motion (Chen et al., 5 Aug 2025).

Among polyp-specific methods, the paper reports that YONA achieves Precision 92.8, Recall 93.8, and F1 93.3 on CVC-ClinicVideoDB, whereas AVPDN reaches 95.9, 94.9, and 95.3 respectively (Chen et al., 5 Aug 2025). This situates AVPDN within a line of video polyp detection methods that includes YONA’s adjacent-frame adaptivity (Jiang et al., 2023), STFT’s multi-frame alignment (Wu et al., 2021), and TSdetector’s temporal-spatial self-correction on YOLOX (Wang et al., 2024).

6. Position in the literature, limitations, and interpretation

AVPDN belongs to a broader progression in colonoscopy video analysis. Earlier work on image-to-video transfer, such as Ivy-Net, treated the problem as domain adaptation from report images to real-time video and emphasized target-domain negative frames and temporal coherence regularization (Zhan et al., 2020). Later methods introduced explicit temporal modules: STFT used proposal-guided deformable convolutions and channel-aware attention (Wu et al., 2021), YONA argued that one adjacent reference frame can be sufficient when foreground and background are handled adaptively (Jiang et al., 2023), and TSdetector combined temporally modulated convolutions, hierarchical temporal integration, and position-aware confidence correction (Wang et al., 2024). AVPDN differs from these in that its “video” character lies mainly in motion-robust and scale-adaptive feature design rather than in explicit adjacent-frame interaction (Chen et al., 5 Aug 2025).

That distinction is also the main limitation. The AVPDN paper explicitly states no temporal consistency loss or sequential supervision, and the current formulation does not explicitly exploit temporal continuity (Chen et al., 5 Aug 2025). It is therefore closer to a domain-specialized robust frame detector for colonoscopy video frames than to a fully temporal video detection architecture. This suggests that its gains come from representation robustness to video artifacts rather than from modeling temporal persistence itself.

Other limitations are also stated or implied in the paper. It does not explain whether the unlabeled portion of LDPolypVideo is used in training, it does not report data augmentation beyond normalization and resizing, and it does not provide FLOPs, parameter counts, or a formal failure-case analysis (Chen et al., 5 Aug 2025). The loss specification is incomplete relative to standard RT-DETR training because classification and assignment details are not documented (Chen et al., 5 Aug 2025). A plausible implication is that reproduction depends on inheriting default RT-DETR machinery while inserting AFIA and SACI.

Within the narrower sense of the term, however, AVPDN names an explicit architecture and contributes a specific thesis: colonoscopy video detection can be improved by learning motion-robust and scale-adaptive representations inside a real-time detector. Its distinctiveness lies in the combination of dense self-attention, ReLU-based sparse self-attention, channel shuffle, and dual-branch dilated context integration, all layered onto RT-DETR without introducing dedicated temporal memory or multi-frame inference (Chen et al., 5 Aug 2025). In that sense, AVPDN marks a branch of the literature in which “adaptive” refers less to explicit temporal aggregation than to representational adaptation to the visual instability of colonoscopy video.