TargetFusionNet: Surgical Triplet Segmentation

• TargetFusionNet is a transformer-based instance segmentation architecture that spatially grounds surgical action triplets with precise instrument, verb, and target predictions.
• It incorporates weak anatomical priors via a gated cross-attention module, effectively fusing visual and tissue segmentation features to enhance recognition accuracy.
• Empirical results on the CholecTriplet-Seg dataset demonstrate significant improvements in mAP metrics, establishing a new baseline for surgical scene understanding.

TargetFusionNet is a transformer-based instance segmentation architecture designed for spatially grounding surgical action triplets, specifically within the context of surgical scene understanding. Its principal innovation is the integration of weak anatomical priors, via a gated cross-attention mechanism, into the process of predicting pixel-accurate instrument masks alongside corresponding action triplets ⟨Instrument, Verb, Target⟩. The architecture builds on Mask2Former, addressing the limitations of frame-level triplet recognition and class activation map-based grounding by providing instance-level, spatially localized instrument-action-target predictions. TargetFusionNet achieves consistent improvements in action triplet segmentation accuracy, as demonstrated on the CholecTriplet-Seg dataset, establishing a new baseline for interpretable, strongly supervised surgical action recognition (Alabi et al., 1 Nov 2025).

1. Input Modalities and Output Structure

TargetFusionNet ingests RGB images $x \in \mathbb{R}^{H \times W \times 3}$ and utilizes both visual and anatomical information:

• Visual Backbone: A ResNet-50 network extracts a set of multi-scale visual feature maps $$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$.
• Weak Anatomy Priors: An off-the-shelf tissue segmentation network (EndoViT) generates per-pixel anatomical logits $L \in \mathbb{R}^{H \times W \times C_t}$ ($C_t = 6$ coarse tissue classes). An anatomy encoder projects $L$ into multi-scale feature maps $$F^t = \{F_l^t \in \mathbb{R}^{H_l \times W_l \times d}\}$$, aligned with $F^v$ in terms of spatial resolution and channel dimensionality.
• Outputs: The network produces a set of $N$ instance masks $M = \{M_k \in [0,1]^{H \times W}\}$, together with a single class $c_k \in \{1,\dots,100\}$ per mask, representing one of the clinically valid triplets.

2. Model Architecture and Data Flow

TargetFusionNet extends the Mask2Former pipeline through the following components and workflow:

1. Visual Feature Extraction: The input image passes through ResNet-50, yielding $$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$0, which feeds into Mask2Former's pixel encoder, producing visual keys ($$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$1) and values ($$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$2).
2. Anatomy Feature Extraction: Anatomy logits $$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$3 are processed by a convolutional downsampling pyramid, yielding $$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$4, which are flattened into anatomy keys ($$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$5) and values ($$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$6).
3. Transformer Decoder: $$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$7 learnable queries $$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$8 are initialized. For each decoder layer $$F^v = \{F_l^v \in \mathbb{R}^{H_l \times W_l \times d} \mid l{=}1\dots4\}$$9:
   • Self-attention is applied: $L \in \mathbb{R}^{H \times W \times C_t}$0.
   • Visual cross-attention: $L \in \mathbb{R}^{H \times W \times C_t}$1.
   • Target-Aware Fusion: A gated cross-attention module fuses anatomical context, yielding $L \in \mathbb{R}^{H \times W \times C_t}$2 (see Section 3).
   • Feedforward network: $L \in \mathbb{R}^{H \times W \times C_t}$3.
4. Prediction Heads:
   • Mask Head: Each final query $L \in \mathbb{R}^{H \times W \times C_t}$4 is projected to a mask embedding $L \in \mathbb{R}^{H \times W \times C_t}$5, used in dynamic convolution with visual pixel features to form $L \in \mathbb{R}^{H \times W \times C_t}$6.
   • Classification Head: A linear projection $L \in \mathbb{R}^{H \times W \times C_t}$7 yields logits over the 100 triplet classes, followed by Softmax probabilities $L \in \mathbb{R}^{H \times W \times C_t}$8.

3. Target-Aware Fusion Module

The distinguishing characteristic of TargetFusionNet is its target-aware fusion mechanism, which enables the incorporation of coarse anatomical priors into each instance query. At each decoder layer, after visual cross-attention, the fusion is formulated as follows:

• Let query embeddings $L \in \mathbb{R}^{H \times W \times C_t}$9, anatomy keys $C_t = 6$0, and anatomy values $C_t = 6$1 (where $C_t = 6$2), the module computes:

$C_t = 6$3

$C_t = 6$4

$C_t = 6$5

Here, $C_t = 6$6 are projection matrices for attention, $C_t = 6$7 computes per-feature gating, $C_t = 6$8 denotes the sigmoid activation, and $C_t = 6$9 is the element-wise product. This gated injection of anatomical context enables the model to regulate the influence of potentially noisy anatomical priors, preventing overreliance on weak or imprecise tissue cues.

4. Supervision and Objective Functions

TargetFusionNet is supervised via composite loss functions tailored to both segmentation and triplet prediction:

• Mask Segmentation Loss: Combines binary cross-entropy and a Dice loss,

$L$0

• Triplet Classification Loss: Cross-entropy over 100 classes,

$L$1

with $L$2 the ground-truth triplet one-hot encoding.

• Multi-Task Variant: Optionally, Mask2Former-style multi-head prediction of instrument, verb, and target uses individual cross-entropy losses for each component (instrument, verb, and target). However, the single-head approach produces the highest $L$3.

5. Training Protocol and Experimental Results

Training uses MMDetection on a single NVIDIA A100 GPU. The visual backbone is pretrained on CholecInstanceSeg, optimizer is AdamW with learning rate and weight decay of $L$4. Augmentations include random flip, scale, and crop; training runs for 300,000 iterations at 1024×1024 resolution with batch size 8.

Empirical evaluation on CholecTriplet-Seg demonstrates substantial improvements:

Method	mAP_I	mAP_V	mAP_T	mAP_IV	mAP_IT	mAP_IVT
RDV-Det (CAM + boxes)	0.09	0.11	0.08	0.08	0.04	0.03
RDV + Mask2Former (sep.)	48.11	32.51	16.29	14.40	11.38	8.73
Mask2Former-Triplet	65.24	45.61	20.75	23.03	16.47	12.23
TargetFusionNet	67.19	46.27	21.55	24.93	17.75	13.47

Ablation experiments show that the gated cross-attention (TargetFusionNet) outperforms early (pixel-level) and late (post-decoder) fusion schemes in mAP metrics, notably for full triplet grounding $L$5.

6. Comparative and Statistical Validation

Fusion strategy ablation confirms that the gated integration of anatomical priors yields the optimal balance for spatial grounding. Early concatenation of priors with RGB yields higher $L$6 but lower overall triplet grounding. Late fusion underperforms. The improvement over Mask2Former-Triplet is statistically significant according to a Wilcoxon signed-rank test ($L$7) over twelve 500-frame subsets (Alabi et al., 1 Nov 2025).

Qualitative analyses (see Fig. 5 in (Alabi et al., 1 Nov 2025)) highlight TargetFusionNet's advantage under challenging scenario such as occlusions and tool clutter. The architecture preserves precise instrument-target associations and demonstrates improved discrimination between visually similar anatomical structures (e.g., cystic duct vs. cystic artery).

7. Technical and Practical Significance

TargetFusionNet advances the paradigm of triplet segmentation by introducing a robust, generalizable mechanism for fusing weak anatomical cues with strong visual features at the instance-query level. The methodological contributions include the formalization of the target-aware fusion module and the demonstration that anatomical priors should be modulated via gated cross-attention rather than concatenation.

This approach enables pixel-accurate, spatially grounded recognition of instrument-action-target relations, improving interpretability and reliability in downstream surgical scene understanding applications. It establishes a unified, strongly supervised framework for the analysis of complex surgical interactions, with implications for automation, education, and intraoperative guidance (Alabi et al., 1 Nov 2025).