CVS-AdaptNet for Automated CVS Recognition

• CVS-AdaptNet is a multi-label adaptation strategy that aligns endoscopic image embeddings with positive and negative text prompts to detect CVS criteria.
• It employs large-scale multi-modal pre-training on PeskaVLP and a contrastive KL-divergence loss, eliminating the need for costly spatial annotations.
• Empirical results on the Endoscapes-CVS201 dataset show a 6.1 point mAP improvement over a ResNet50 baseline, highlighting its efficacy in complex surgical settings.

CVS-AdaptNet is a multi-label adaptation strategy for automated Critical View of Safety (CVS) recognition in laparoscopic cholecystectomy that leverages large-scale multi-modal surgical pre-training and natural-language descriptions rather than expensive spatial annotations. It was introduced to address the fact that CVS assessment remains complex even for experts, while existing multi-modal surgical models exhibit a significant performance gap in zero-shot evaluation for this task. The method adapts PeskaVLP, a vision–language surgical foundation model, by aligning endoscopic image embeddings with positive and negative textual descriptions of each of the three Strasberg CVS criteria, and it is trained with a contrastive KL-divergence loss on the Endoscapes-CVS201 dataset (Baby et al., 7 Jul 2025).

1. Clinical task and problem formulation

CVS recognition in this setting is formulated as a fine-grained multi-label classification problem with $K=3$ criteria. Each training image $x_j$ is associated with a binary label vector $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$, where $y_{ji}\in\{0,1\}$ indicates absence or presence of criterion $i$ (Baby et al., 7 Jul 2025).

This formulation is important because the target is not a conventional multi-class decision. The supplied description explicitly contrasts CVS recognition with many existing multi-modal models that are primarily adapted for multi-class classification. In this case, each image may satisfy multiple criteria simultaneously, so the learning objective must preserve criterion-wise binary supervision rather than collapse the task into a single categorical label. The same source also emphasizes that the approach does not use bounding boxes or masks, which distinguishes it from spatial annotation-based pipelines.

Traditional CVS recognition models are described as vision-only systems that learn from costly, labor-intensive spatial annotations. CVS-AdaptNet is positioned as an alternative that uses text both during training and inference. This suggests a shift from explicit spatial supervision toward representation alignment in a shared image–text space, with the aim of reducing dependence on expensive annotation protocols.

2. Prompt-based multi-label adaptation

At the core of CVS-AdaptNet is criterion-specific alignment between image embeddings and textual prompts. For each criterion $i$, the framework uses positive prompts $\{y_{ji}^+\}$ describing that criterion as present and negative prompts $\{y_{ji}^-\}$ describing it as absent. The supplied examples are “Seeing two structures cystic duct and cystic artery in the image” for presence and “A general medical image not showing cystic duct or cystic artery” for absence (Baby et al., 7 Jul 2025).

A large-LLM, exemplified by ChatGPT-4, is used to generate diverse paraphrases for these prompts. The stated purpose is to increase robustness to prompt wording. Images and text are embedded by an image encoder $g_I$ and a text encoder $g_W$, both initialized from PeskaVLP:

$x_j$0

For each image–criterion pair, the text feature is selected according to the ground-truth binary label:

$x_j$1

The design is therefore label-conditioned at the prompt-selection stage. Rather than using only positive textual supervision, CVS-AdaptNet explicitly contrasts presence and absence for every criterion. In the terms of the source, this enhances fine-grained, binary classification across multiple labels by aligning image embeddings with textual descriptions of each CVS criterion using positive and negative prompts. A plausible implication is that the negative prompts regularize the decision boundary for clinically ambiguous frames, especially when a criterion is not visible but the scene remains surgically plausible.

3. Alignment objective and optimization

The image–text similarity function is the cosine similarity

$x_j$2

The framework then defines normalized alignment probabilities over a batch of text prompts, including both positive and negative prompts for all images. The loss for criterion $x_j$3 on image $x_j$4 is a symmetric KL-divergence contrastive objective:

$x_j$5

where the ground-truth match indicator $x_j$6 is $x_j$7 for the selected positive or negative prompt and $x_j$8 otherwise. The total batch loss sums over all criteria:

$x_j$9

The training pipeline uses Endoscapes-CVS201, with 11,090 laparoscopic frames split into 6,960 train, 2,331 validation, and 1,799 test images. The backbone is PeskaVLP, with a ResNet-50 vision encoder and a BioClinicalBERT text encoder. Images are resized to $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$0, center-cropped to $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$1, and augmented with RandAugment $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$2; text is augmented through LLM paraphrasing. Optimization uses Adam with initial learning rate $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$3 for vision and text parameters, $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$4 for the learnable temperature $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$5, and $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$6 initialized at $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$7. Training is performed with batch size $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$8 for $\mathbf{y}_j=(y_{j1},y_{j2},y_{j3})$9 epochs on a single RTX A5500 GPU in approximately four hours, with cosine annealing and best-model selection by validation mAP (Baby et al., 7 Jul 2025).

These implementation details indicate that the method is not merely a prompt-engineering layer on top of a frozen model. It is a fine-tuning strategy for a surgical foundation model under a multi-label contrastive objective, with the learnable temperature and batchwise alignment probabilities functioning as integral parts of the optimization.

4. Inference schemes and interpretability of image–text alignment

After fine-tuning, CVS-AdaptNet supports three text-specific inference modes. The first is standard inference: an image embedding $y_{ji}\in\{0,1\}$0 is compared with clinician-selected text features $y_{ji}\in\{0,1\}$1, one per criterion, and prediction is given by

$y_{ji}\in\{0,1\}$2

The second is positive-negative inference. For each criterion $y_{ji}\in\{0,1\}$3, one uses sets $y_{ji}\in\{0,1\}$4 and $y_{ji}\in\{0,1\}$5 of positive and negative prompts, computes the similarities

$y_{ji}\in\{0,1\}$6

and normalizes them as

$y_{ji}\in\{0,1\}$7

The third is multi-class inference. Here the method generates $y_{ji}\in\{0,1\}$8 composite descriptions enumerating combinations of the three criteria, computes similarities to each composite text, applies a softmax over the eight descriptions, and then aggregates the resulting probabilities back to the three binary criteria:

$y_{ji}\in\{0,1\}$9

The source states that these inference modes help in analysing the image-text alignment. That is especially explicit for positive-negative contrasts, where the model can be queried not only for the compatibility of an image with a positive clinical statement but also for its compatibility with the corresponding negative statement. This makes inference more flexible than a purely vision-driven classifier, although the supplied data does not claim formal interpretability guarantees.

5. Empirical results on Endoscapes-CVS201

The reported evaluation uses mean Average Precision (mAP) and per-criterion AP for the three CVS criteria, averaged over three runs. CVS-AdaptNet improves over the ResNet50 image-only baseline and is competitive with a stronger self-supervised image-only baseline, while remaining below the upper bound reported for a segmentation-based graph method (Baby et al., 7 Jul 2025).

Method	Setting	mAP
ResNet50 (ImageNet)	Image-only baseline	51.5
ResNet50-MoCoV2	Image-only baseline	57.4
PeskaVLP-vision	Vision-only, PeskaVLP init	48.9 ± 2.3
CVS-AdaptNet	Standard inference	57.6 ± 0.2
CVS-AdaptNet	Positive-Negative	56.8 ± 0.4
CVS-AdaptNet	Multi-Class	57.6 ± 0.3
Zero-shot PeskaVLP	No adaptation	≈ 26.6
LG-CVS*	Upper bound with segmentation masks	67.3

For the standard inference mode, the per-criterion AP values are $i$0 for C1, $i$1 for C2, and $i$2 for C3. The ResNet50 (ImageNet) baseline records C1 $i$3, C2 $i$4, and C3 $i$5; ResNet50-MoCoV2 records C1 $i$6, C2 $i$7, and C3 $i$8. The key gain emphasized in the source is

$i$9

described as an improvement of 6 points over the ResNet50 baseline.

The zero-shot result is also notable: zero-shot PeskaVLP achieves approximately $i$0 mAP, compared with random $i$1. This addresses a common misconception that a pre-trained multi-modal surgical model can simply be transferred to fine-grained CVS assessment without adaptation. In the supplied results, zero-shot transfer remains far below the fine-tuned CVS-AdaptNet variants. At the same time, the method still trails the segmentation-mask upper bound $i$2, showing that annotation-free multi-modal alignment does not yet eliminate the performance advantage of pipelines using explicit pixel-wise structure.

6. Limitations, future directions, and terminological overlap

The advantages stated for the multi-modal formulation are threefold: text prompts inject rich clinical semantics without costly bounding boxes or masks; flexible, queryable inference helps in ambiguous cases; and large-scale pre-training on surgical lecture videos allows better generalization than random re-training (Baby et al., 7 Jul 2025).

The limitations are equally explicit. CVS-AdaptNet still trails state-of-the-art segmentation-based graph methods, depends on the quality and coverage of LLM-generated prompts, has been evaluated on a single CVS dataset, and does not exploit spatial localization of cues such as region proposals. These constraints delimit the scope of current claims. In particular, cross-dataset generalization is identified as remaining to be tested, so broader robustness should not be assumed from the available evidence.

The future directions listed in the source include combining image–text alignment with weak localization through attention maps or pseudo-masks, extending the framework to other multi-label surgical tasks such as anatomy presence and risk factors, exploring advanced prompt-tuning or adapter modules such as DualCoOp and MMA once architectural compatibility is resolved, scaling to larger vision–language backbones and additional datasets, and investigating active or curriculum learning to refine ambiguous labels.

A separate point concerns nomenclature. The supplied material also includes SAIL, a test-time adaptation framework that introduces a lightweight learnable AdaptNet attached to a frozen vision–LLM and is “also referred to here as CVS-AdaptNet” in that summary (Chen et al., 3 Jun 2025). In SAIL, the frozen VLM and AdaptNet are combined by a confidence-based interpolation weight, AdaptNet is trained at inference time with a self-supervised alignment loss, and a gradient-aware reset strategy uses a gradient drift indicator to mitigate catastrophic forgetting. This suggests a terminological overlap between the surgical CVS-AdaptNet of multi-label CVS recognition and the broader AdaptNet component used in SAIL, rather than a single unified method.