Cataract-1K: Surgical Video Dataset

• Cataract-1K is a comprehensive video dataset of 1,000 cataract surgeries with multi-modal annotations for phase recognition, semantic segmentation, and irregularity detection.
• It uses rigorous, dual-review annotation protocols across temporal, anatomical, instrumental, and event modalities to ensure high-quality benchmarks for computer-assisted surgical analysis.
• Benchmarking experiments reveal robust performance, with bidirectional RNNs achieving up to 95.37% F1 on key phases, while highlighting challenges in cross-domain adaptation.

Cataract-1K is the largest publicly reported video dataset of phacoemulsification cataract surgery, developed to provide high-fidelity benchmarks for the computer-assisted analysis of ophthalmic surgery workflows. Designed to address core challenges in surgical scene understanding, intraoperative phase recognition, and the detection of procedural irregularities, Cataract-1K enables reproducible, multi-layered evaluation of vision-based algorithms. The resource incorporates diverse annotation modalities—temporal, anatomical, instrumental, and event-based—supporting algorithmic research in surgical step prediction, semantic scene parsing, irregularity detection, and cross-domain instrument segmentation (Ghamsarian et al., 2023).

1. Dataset Composition and Structure

Cataract-1K consists of 1,000 cataract surgery videos, each recorded at 25 frames per second, with a native frame resolution of 512 × 324 pixels, acquired at Klinikum Klagenfurt (Austria) between 2021 and 2023. The total raw duration exceeds 1,000,000 seconds (approximately 278 hours). It comprises distinct, highly curated subsets for three principal tasks:

• Phase Recognition: 56 representative (“regular”) surgeries, each captured at 1024 × 768 pixels and 30 fps, are fully annotated on a per-frame basis with 13 discrete action phases (12 procedural phases plus “idle”).
• Semantic Segmentation: 2,256 frames from 30 surgeries, sampled every 5 seconds, are polygonally annotated via Supervisely into 3 anatomical classes (Iris, Pupil, Intraocular Lens) and 9 instrument classes (Slit/Incision Knife, Gauge, Spatula, Capsulorhexis Cystotome, Phacoemulsifier Tip, Irrigation–Aspiration, Lens Injector, Capsulorhexis Forceps, Katena Forceps).
• Irregularity Event Detection: Temporally localized annotations for two event types—pupil contraction (miosis) during phacoemulsification (rapid shrinkage in under 1 second), and intraoperative intraocular lens (IOL) rotations (fast, clockwise, <7 seconds)—are timestamped for each occurrence.

Within the phase recognition subset, phase usage is non-uniform: for instance, “Phacoemulsification” comprises 28.72% of annotated frames, while “Incision” comprises only 2.1% (Ghamsarian et al., 2023, Shah et al., 29 Jul 2025). All annotation files (CSV for phase boundaries, Supervisely JSON/COCO for masks) and supporting code are released under CC BY 4.0.

2. Annotation Protocols and Label Quality

The annotation process is multi-layered and quality-controlled:

• Temporal Phase Labels: Two expert annotators (a surgeon and an engineer) follow a rigid phase-by-phase schema, with frame-level start and end recordings for all 13 steps in the 56-video subset; provided as CSV.
• Scene Segmentation: Expert polygons are constructed for each semantic class within the Supervisely platform, then exported as JSON. All labels are subject to ophthalmologist review.
• Irregularity Events: Event timing (onset/offset) is determined by consensus between two surgeons. No formal inter-annotator agreement statistics (e.g., κ) are reported, but dual review is implemented to ensure fidelity.

This rigorous protocol enables high annotation reliability, permitting robust benchmarking of both temporal and spatial models (Ghamsarian et al., 2023).

3. Benchmarking Experiments and Evaluation

Cataract-1K establishes standardized metrics and extensive baselines for each targeted algorithmic task.

Phase Recognition: Frame-sequences of 10 frames are drawn from overlapping 3-second video clips (32 train, 24 test). Models utilize ImageNet-pretrained CNN backbones (VGG16, ResNet50) with recurrent heads (LSTM/GRU or bidirectional variants). Training uses binary cross-entropy, Adam optimizer (lr = 0.001), and 0.5 dropout, with the last four CNN layers frozen. Evaluation is per-phase accuracy and F1 score: $$\mathrm{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN},\quad \mathrm{F1} = 2\,\frac{\mathrm{precision}\times \mathrm{recall}}{\mathrm{precision} + \mathrm{recall}}$$ Bidirectional RNNs yield 2–4% higher F1 than unidirectional; the best model (ResNet50–BiGRU) achieves an F1 of 95.37% on Phacoemulsification and the lowest (≈57%) on Viscoelastic/AC-Flushing (Ghamsarian et al., 2023).

Semantic Segmentation: Multiple architectures (DeepPyramid, Adapt-Net, UNet++, ReCal-Net, CPFNet, CE-Net, FED-Net, scSENet, DeepLabV3+, UPerNet, U-Net+) are evaluated under a 5-fold patient-wise split, with standard augmentations. Loss is a mixture of BCE and soft Dice; metrics are mean Intersection over Union (IoU) and Dice coefficient: $$\mathrm{IoU} = \frac{TP}{TP + FP + FN},\quad \mathrm{Dice}= \frac{2\,TP}{2\,TP + FP + FN}$$ DeepPyramid (VGG16) attains the best mean IoU (88.4%) on anatomical classes; instrument segmentation remains more challenging (mean Dice ≈78%) due to transparency and small or blurred instrument tips (Ghamsarian et al., 2023).

Irregularity Detection: Event labels support temporal localization and prediction lead time experiments.

4. Cross-Domain Transfer and Domain Adaptation

To study domain generalization, instrument segmentation models trained on Cataract-1K are tested zero-shot on CaDIS, a public cataract segmentation benchmark. Adapt-Net achieves an IoU of 74.4 ± 3.3% on Cataract-1K but only 49.6 ± 1.5% on CaDIS (Dice: 81.5% → 61.7%). DeepPyramid exhibits a similar decline. This severe domain gap is attributed to shifts in operating room environments and device characteristics, motivating research in semi-supervised and domain-adaptation methods (Ghamsarian et al., 2023).

5. Dataset Splits, Class Distribution, and Statistical Features

Within the 56 video “labeled subset” used for most learning benchmarks, each frame is annotated as one of 13 surgical steps (12+idle) (Shah et al., 29 Jul 2025). For machine learning workflows:

• All videos are downsampled to 1 fps and frames resized (typically to 250 × 250 or 224 × 224 for input to networks).
• Videos are split: 25 for training, 7 for validation, and 24 for test.
• The frequency vector $\{p_s\}$ for steps is non-uniform (steps like phacoemulsification and IOL insertion are nearly ubiquitous; rarer maneuvers appear infrequently).
• Evaluation metrics on this subset comprise overall accuracy, mean per-class precision, recall, and Jaccard index, computed over all test frames:

$$\mathrm{Acc} = \frac{1}{T_{\mathrm{test}}} \sum_{t=1}^{T_{\mathrm{test}}} \mathbf{1}(\hat y_t=y_t)$$

$$\mathrm{Precision} = \frac{1}{C} \sum_{s=1}^{C} \frac{\mathrm{TP}_s}{\mathrm{TP}_s + \mathrm{FP}_s}$$

$$\mathrm{Recall} = \frac{1}{C} \sum_{s=1}^{C} \frac{\mathrm{TP}_s}{\mathrm{TP}_s + \mathrm{FN}_s}$$

$$\mathrm{Jaccard} = \frac{1}{C} \sum_{s=1}^{C} \frac{\mathrm{TP}_s}{\mathrm{TP}_s + \mathrm{FP}_s + \mathrm{FN}_s}$$

This evaluation protocol provides granular insight into model performance across differing class prevalences (Shah et al., 29 Jul 2025).

6. Usage Guidelines, API, and Applications

All ground-truth annotations (CSV/JSON/COCO formats) and code (mask generation, phase extraction, training scripts) are made available under a permissive license. Python APIs for mask conversion, data extraction, and metric reporting are provided, with recommended evaluation setups:

• Phase recognition: One-vs-rest, 3-second clips at 30 fps, using 5-fold video splits or the standard train/test split.
• Segmentation: 5-fold patient-wise cross-validation, with per-class mean and standard deviation of IoU and Dice.
• Irregularity detection: Accurate timing and lead time reporting.

A Docker image ensures reproducible benchmarks. Applications of Cataract-1K encompass real-time intraoperative alerts (phase-aware tool tracking, miosis warnings), skill assessment, relevance-driven video summarization, and automated surgical reporting. Its multi-modal annotation strategy is intended to stimulate advances in intelligent, context-aware surgical systems (Ghamsarian et al., 2023).

7. Context, Related Work, and Limitations

Cataract-1K’s scale and annotation density surpass previous cataract surgery datasets. Prior work in cataract video workflow analysis was constrained by small sample sizes, incomplete step coverage, or a lack of detailed mask-level annotations. The incorporation of temporally-precise irregularity events and multi-domain test protocols further distinguishes Cataract-1K as an enabling tool for research in machine learning for ophthalmic surgery (Ghamsarian et al., 2023).

A plausible implication is that while the dataset significantly reduces the bottleneck of high-quality data for model development, the persistent domain gap with other datasets (e.g., CaDIS) and the class imbalance in certain phases indicate that real-world deployment will require further algorithmic innovations (adaptive sampling, domain adaptation, active learning) (Shah et al., 29 Jul 2025).

Cataract-1K represents the current state-of-the-art corpus for computer vision–based analysis of cataract surgery workflow, providing both strong labeled benchmarks and a platform to address open research questions in surgical AI.