Data-Efficient Surgical Phase Segmentation in Small-Incision Cataract Surgery: A Controlled Study of Vision Foundation Models

Abstract: Surgical phase segmentation is central to computer-assisted surgery, yet robust models remain difficult to develop when labeled surgical videos are scarce. We study data-efficient phase segmentation for manual small-incision cataract surgery (SICS) through a controlled comparison of visual representations. To isolate representation quality, we pair each visual encoder with the same temporal model (MS-TCN++) under identical training and evaluation settings on SICS-155 (19 phases). We compare supervised encoders (ResNet-50, I3D) against large self-supervised foundation models (DINOv3, V-JEPA2), and use a cached-feature pipeline that decouples expensive visual encoding from lightweight temporal learning. Foundation-model features improve segmentation performance in this setup, with DINOv3 ViT-7B achieving the best overall results (83.4% accuracy, 87.0 edit score). We further examine cataract-domain transfer using unlabeled videos and lightweight adaptation, and analyze when it helps or hurts. Overall, the study indicates strong transferability of modern vision foundation models to surgical workflow understanding and provides practical guidance for low-label medical video settings. The project website is available at: https://sl2005.github.io/DataEfficient-sics-phase-seg/

• The paper presents a controlled study showing that self-supervised vision models (e.g., DINOv3) outperform traditional supervised encoders, achieving up to 83.4% accuracy.
• The methodology decouples expensive frame-level feature extraction from lightweight temporal modeling by using a frozen encoder and a fixed MS-TCN++ head.
• The work demonstrates that lightweight domain adaptation (CataractFT with LoRA) can further improve performance in low-label settings while balancing temporal consistency.

Data-Efficient Surgical Phase Segmentation in Small-Incision Cataract Surgery Using Vision Foundation Models

Introduction

This study addresses the critical problem of surgical phase segmentation in manual small-incision cataract surgery (SICS) scenarios characterized by limited labeled video data. Phase segmentation has direct utility for intraoperative guidance, retrospective workflow analysis, and surgical skill assessment, yet progress is hampered by high annotation costs and data scarcity. The authors propose an experimental protocol that isolates visual encoder quality by decoupling expensive visual feature extraction from downstream sequence modeling, utilizing a frozen encoder pipeline with a fixed MS-TCN++ temporal head. Both supervised (ResNet-50, I3D) and large-scale self-supervised (DINOv3, V-JEPA2) vision models are benchmarked on the SICS-155 dataset. Furthermore, the study investigates cataract-domain transfer through unsupervised continuation and lightweight LoRA adaptation, quantifying when these steps offer practical improvement.

Figure 1: Controlled study pipeline. Encoders are frozen after optional cataract-domain SSL continuation and LoRA adaptation, features are cached once, and the same MS-TCN++ head is trained for all encoders.

Experimental Protocol and Pipeline

The main innovation of the pipeline is a deployment-oriented factorization: once per-frame representations are computed via a frozen visual backbone, only the lightweight MS-TCN++ temporal head is optimized over the low-dimensional cached features. This design supports controlled comparison and matches deployment settings where computational cost and label availability are significant constraints. The comparison encompasses ResNet-50 (ImageNet), I3D (Kinetics), DINOv3 (ViT-B, ViT-L, ViT-7B), and V-JEPA2 (ViT-L, ViT-g). For DINOv3, each frame is encoded independently; for V-JEPA2, sliding-window clip features are linearly interpolated to generate frame-level embeddings.

Evaluation is performed using stratified 5-fold cross-validation on SICS-155, which comprises 155 annotated SICS procedures spanning 19 surgical phases—characterized by significant long-tail imbalance (Figure 2). The identical splits, optimization schedule, and deterministic computation are maintained for all experiments, ensuring any observed differences stem from representation quality alone.

Figure 2: SICS-155 phase distribution at frame level. The long-tail imbalance motivates representation learning approaches that remain robust under limited labels.

Encoder Performance and Representation Scaling

Self-supervised DINOv3 exhibits superior phase segmentation performance relative to traditional supervised encoders under all metrics, with DINOv3 ViT-7B reaching 83.4% accuracy (mean over folds), 76.5% macro-F1, and 87.0 edit score. DINOv3 ViT-L, while slightly lower, provides a more favorable accuracy/VRAM consumption trade-off. Within the DINOv3 family, results indicate a monotonic scaling effect, where larger models achieve systematically higher accuracy and edit scores. Importantly, the performance gap between DINOv3 ViT-L/ViT-7B and I3D exceeds interfold variability, establishing the efficacy of large-scale self-supervised vision models for dense sequence labeling in SICS.

V-JEPA2, despite being pretrained for video modeling, underperforms relative to DINOv3 and I3D. The authors attribute this to two confounding factors: (1) the stride-4 extraction and interpolation potentially oversmooth boundaries, impairing temporal discrimination, and (2) the domain shift between generic web-scale video pretraining and operating room videos.

Qualitative analysis reflects these findings: DINOv3 reduces visually choppy predictions and phase transition errors present in supervised models. Representative temporal segmentation outputs corroborate quantitative improvements (Figure 3).

Figure 3: Example phase segmentation: ground-truth, supervised baseline, foundation model alternative, and top-performing foundation model. Stronger representations reduce fragmented predictions.

CataractFT: Domain Transfer and Lightweight Adaptation

To address low-label settings, the authors introduce CataractFT, leveraging unsupervised pretraining on 1,000 unlabeled cataract videos followed by optional LoRA adaptation on limited annotated cataract procedures. This adaptation is evaluated across multiple DINOv3 and V-JEPA2 backbones.

Results show that the benefit of CataractFT is both backbone- and metric-dependent. For DINOv3 ViT-B, LoRA augmentation meaningfully improves accuracy (+2.86 points), macro-F1 (+2.87), and PR-AUC, with moderate edit score gains (+1.66), relative to feature extractors only continued on the unlabeled cataract data. In contrast, the more capable DINOv3 ViT-L gains marginally or regresses, indicating that highly expressive encoders pretrained on large-scale web data already capture much transferable structure, and further adaptation can lead to partial overfitting or reduced temporal consistency.

For segment-level metrics, LoRA adaptation improves framewise discrimination but can degrade edit score, consistent with sharper but sometimes less temporally smooth predictions—particularly for phases with brief or ambiguous transitions. Per-phase analysis reveals that LoRA offers most value in instrument-centric, visually distinctive phases (e.g., tunnel suture, cautery), but can negatively affect amorphous or temporally diffuse phases (Figure 4).

Figure 4: Per-phase change from CataractFT + LoRA relative to SSL-only continuation. Gains are concentrated in phases with discriminative visual cues; some phases exhibit reduced overlap after adaptation.

Practical Implications and Deployment Considerations

The study presents strong evidence that self-supervised vision foundation models are effective drop-in alternatives to supervised encoders for surgical phase segmentation, especially under realistic compute and annotation constraints. Deploying ViT-L in place of ViT-7B may offer a more practical accuracy/compute trade-off, especially in low-resource settings typical of SICS practice. Moreover, LoRA-based adaptation pipelines like CataractFT should be considered when annotated data is extremely scarce and domain shift exists, but practitioners should carefully monitor not just frame-level improvements but also sequence-level temporal consistency, as adaptation can exacerbate spurious transitions or fragment predictions.

Theoretical and Future Directions

The results expose the importance of evaluating both frame-centric and sequence-level metrics in surgical video understanding, as gains in representation sharpness can reveal undesirable behaviors in segment-level structure. The controlled pipeline facilitates clear benchmarking and supplies a robust protocol for representation scaling studies. However, current findings also indicate that the benefits of domain adaptation (e.g., CataractFT) saturate quickly with model capacity, and that further research is required to disentangle instrument-specific cues from holistic scene understanding.

Future work should expand evaluation across additional surgical domains, develop adaptation strategies that explicitly regularize for temporal smoothness, and consider flexible (non-frozen) temporal heads capable of leveraging backbone adaptation. Investigating the intersection of foundation model representations and advanced temporal architectures remains a promising direction, particularly in the context of annotation efficiency and real-world deployment.

Conclusion

This work provides a systematic, controlled characterization of representation quality for surgical phase segmentation under label scarcity. Large-scale self-supervised visual encoders offer significant advantages over conventional supervised models in SICS, while lightweight adaptation pipelines yield further improvements primarily in less expressive backbones and for visually distinctive surgical phases. The study supplies actionable guidance for deploying phase segmentation systems in low-label and resource-constrained environments, and motivates additional research into representation adaptation, temporal modeling, and domain generalization for surgical workflow analysis.