BleedOrigin-Net: Deep Learning for Bleeding Localization

• BleedOrigin-Net is a dual-stage deep learning framework for dynamic bleeding source localization in ESD, integrating both detection and tracking modules.
• It employs transformer-based temporal modeling, multi-domain gated attention, and confidence-based key-frame memory to robustly identify bleeding events amidst occlusions.
• The method leverages the BleedOrigin-Bench dataset to achieve state-of-the-art accuracy in challenging clinical conditions, providing actionable cues for hemostatic intervention.

BleedOrigin-Net is a dual-stage deep learning framework specifically designed for dynamic bleeding source localization during Endoscopic Submucosal Dissection (ESD) procedures. Addressing the critical clinical need for rapid, precise spatial localization and continuous tracking of bleeding sources—where manual visual identification is severely impeded by dynamic occlusions, blood pooling, and scene disturbances—BleedOrigin-Net introduces novel detection and tracking strategies in conjunction with the first comprehensive annotated ESD bleeding dataset, BleedOrigin-Bench. The methodology integrates transformer-based temporal modeling, multi-domain attention, confidence-based key-frame memory, and parameter-efficient adaptation, achieving state-of-the-art accuracy on both detection and tracking tasks in a challenging clinical domain (Xu et al., 20 Jul 2025).

1. Dual-Stage Framework: Detection and Tracking

BleedOrigin-Net comprises two primary components: BleedOrigin-Detect and BleedOrigin-Track. The former is tasked with temporally localizing the first bleeding frame (“onset”) and spatially pinpointing the source in that frame, while the latter tracks the source coordinate across subsequent frames, robust to obscuration, motion, and appearance changes.

• BleedOrigin-Detect utilizes a sliding window approach to process endoscopic video. Each window (of size $N=60$ frames) is analyzed to predict the bleeding onset time $t_{\mathrm{bleed}}$ and corresponding spatial coordinate $(x^*,y^*)$.
• BleedOrigin-Track initializes a transformer-based online point tracker using $(x^*,y^*)$, producing a continuous trajectory $\{P_t\}$ for all $t > t_{\mathrm{bleed}}$.

This sequential approach models the full clinical workflow from event detection to continuous spatial monitoring, providing actionable cues for hemostatic intervention.

2. Architecture and Technical Innovations

BleedOrigin-Detect

• Backbone: Perception Encoder (PE-Spatial-448) generates multi-channel feature maps.
• Red-Mask Branch extracts raw red-channel intensity, exploiting the spectral signature of blood without parameterized learning.
• Heat-map Branch employs convolutional layers and upsampling to produce spatial probability maps of bleeding likelihood.
• Multi-Domain Gated Attention (MDG) fuses RGB, HSV, and optical-flow domain features, gated to prioritize salient cues even in presence of glare or motion artifacts.
• Multi-Domain Confidence-based Frame Memory (MDCFM) selects “clean” key-frames by evaluating RGB-HSV and flow-based confidence metrics; this mitigates errors introduced by blood occlusion and surgical tool interference.
• Temporal Localization is accomplished using a 4-layer transformer encoder supplemented by positional encodings. The network outputs both an onset location $\theta\in[0,1]$ and a confidence value $S_{\mathrm{conf}}$, selecting the earliest frame index $t_{\mathrm{bleed}}$ with $S_{\mathrm{conf}}>0.5$ within each window.
• Spatial Localization leverages a multi-scale feature pyramid network (FPN) over heatmap, red-mask, and perception encoder outputs, producing a fused attention map whose global maximum identifies $t_{\mathrm{bleed}}$0.

BleedOrigin-Track

• Backbone: Transformer-based online point tracker (Track-On).
• Pseudo-label Generation: Augments sparse manual annotations by extracting XFeat keypoints (within radius $t_{\mathrm{bleed}}$1, match score $t_{\mathrm{bleed}}$2) around ground-truth, propagating them over 30 frames, and refining via Kalman smoothing.
• Chromatic Guidance: Concatenates red-mask with visual features, informing transformer attention with domain-specific spectral cues.
• Parameter-Efficient Fine-Tuning (LoRA): Only attention and MLP adapter ranks are updated, reducing overfitting and improving domain adaptation with minimal parameter update.

3. Mathematical Formulation and Loss Functions

Key operations and supervision mechanisms are mathematically specified:

• Keyframe selection (MDCFM):

$t_{\mathrm{bleed}}$3

• Temporal Localization Loss combines binary cross-entropy and mean squared error over normalized onset position:

$t_{\mathrm{bleed}}$4

• Spatial Localization Loss blends MSE over heatmaps and pseudo-labels, plus Huber loss on point coordinate prediction.
• Tracking Loss uses a weighted sum of Huber distance to ground-truth and to propagated pseudo-labels.

4. Dataset and Annotation Pipeline

BleedOrigin-Bench is constructed from 44 ESD clinical procedures (Qilu Hospital), with the following characteristics:

• Frames: 106,222 at 1 fps, representing 485 bleeding clips.
• Annotations: 1,771 expert-annotated bleeding sources; pseudo-labels supplement with 39,755 coordinates derived from the tracking pipeline.
• Coverage: Data sampled across 8 anatomical sites (e.g., gastric antrum, duodenum) and 6 clinically challenging scenarios (including obscured view, flushing, smoke, light reflection, jitter, and instrument interference).
• Annotation methodology: Dual independent clinical team annotation, followed by majority-voting and researcher quality-filtering.
• Dense-subset validation: Three 15 s+ videos annotated at 30 fps, reserved solely for validation.

5. Training Protocol and Data Organization

• Splits: Patient-level 4:1:1 split for training, validation, and test; no patient overlap.
• Augmentation: Random frame skipping (0–60 frames) ensures bleeding onsets are encountered at random positions during training.
• Optimization (detection): Trained for 500 epochs, learning rate $t_{\mathrm{bleed}}$5.
• Tracking Schedules:
  • Short-clip (31 frames, 100 epochs, $t_{\mathrm{bleed}}$6 learning rate)
  • Long-clip (61–301 frames, 100 epochs, $t_{\mathrm{bleed}}$7)
  • LoRA adapters adjusted in attention/MLP only.

6. Performance and Comparative Evaluation

Detection and Tracking Metrics

Task	Metric	YOLOv12-s	BleedOrigin-Detect/Track
Initial Bleeding Frame Detection	$t_{\mathrm{bleed}}$8 frame accuracy	86.2%	96.9%
Initial Bleeding Source Localization ($t_{\mathrm{bleed}}$9 px)	Pixel-level accuracy	61.6%	70.2%
Continuous Point Tracking ($(x^*,y^*)$0 px)	Pixel-level accuracy	85.6% (Track-On)	96.1%

BleedOrigin-Net consistently outperforms both widely-used object detectors (YOLOv12-s), transformer trackers (Track-On, CoTrackerV3), and modern multimodal LLMs (ChatGPT-4o, Claude-3.5, Gemini, Qwen2.5-VL; all $(x^*,y^*)$1 at 100 px), with markedly superior localization/tracking under clinical conditions (Xu et al., 20 Jul 2025).

7. Clinical and Technical Challenges, Limitations, and Future Directions

Technical Solutions to Domain Challenges:

• Dynamic occlusion and obstruction: MDCFM selectively identifies and preserves clean reference frames, filtering out occluded/noisy views.
• Lighting and appearance changes: MDG’s RGB/HSV/flow fusion adaptively focuses attention on robust blood cues, overcoming glare and water flushing artifacts.
• Annotation sparsity: Pseudo-label propagation and Kalman-based smoothing enable dense supervision from limited ground truth.
• Domain adaptation: LoRA-based parameter-efficient fine-tuning stabilizes transformer adaptation with minimal parameter overhead.

Known Limitations:

• Data originates from a single medical center; lacks multi-institutional validation.
• Algorithm supports single-bleed tracking only; does not generalize to simultaneous multi-source bleeding.
• Only 2D spatial localization is available; no direct depth estimation is performed.
• Key-frame memory refresh is periodic, not content-adaptive.
• Real-time robotic hemostatic actuation is not integrated.

Planned Extensions:

• Expansion to multi-institutional datasets and multi-bleed scenarios.
• Integration of stereo or structured-light imaging for depth (3D) localization.
• Exploration of adaptive memory mechanisms and robotic actuation for closed-loop operation.

8. Significance and Future Directions

BleedOrigin-Net establishes a new technical paradigm for AI-assisted, real-time bleeding source localization and monitoring in ESD, systematically addressing unique challenges inherent to surgical endoscopy. The accompanying BleedOrigin-Bench provides a robust platform for reproducible evaluation and further research toward closed-loop, autonomous hemorrhage management in minimally invasive surgery (Xu et al., 20 Jul 2025).