Oncology Contouring Copilot (OCC)

• Oncology Contouring Copilot (OCC) is an intelligent system integrating machine learning, computer vision, and human-in-the-loop methods to automate and standardize tumor and organ-at-risk delineation.
• It employs multi-stage pipelines combining feature extraction, deep learning segmentation, and interactive revision workflows to enhance accuracy and efficiency in radiotherapy planning.
• OCC systems reduce manual workload, improve contour consistency, and support clinical integration by incorporating uncertainty quantification and multimodal data processing.

An Oncology Contouring Copilot (OCC) is a class of intelligent, semi-automatic or automatic systems designed to assist radiation oncologists and medical imaging professionals in the delineation of tumors and organs-at-risk (OAR) for oncological treatment planning. OCCs leverage contemporary advancements in machine learning, computer vision, user interaction, and uncertainty quantification to reduce manual workload, increase efficiency, standardize contouring practices, and facilitate communication between human experts and AI models across a range of modalities and clinical contexts.

1. Conceptual Framework and System Architectures

OCC systems are implemented as multi-stage pipelines or interactive platforms that combine image analysis algorithms, user interfaces, and quality assurance modules. Architectures include:

• Feature-based and Semi-Automated Methods: Early-stage OCCs extract keypoints using descriptors such as SURF or FAST, augment feature vectors with spatial coordinates (distance to user-defined center), classify keypoints using SVMs, and represent tumor regions with geometric primitives (e.g., ellipsification) (Gangeh et al., 2017).
• Automatic 2D/3D Segmentation: Deep learning (DL)–based OCCs utilize 3D U-Net variants or region-growing with level-set refinement to produce volumetric tumor contours, often integrating pre-processing (e.g., contrast adjustment, cropping) for computational efficiency (Chen et al., 2019, Astaraki et al., 2023).
• Interactive and Revision Workflows: Modern frameworks support clinician-in-the-loop revision; initial automatic segmentation is refined iteratively via click-based feedback or similar input, with each interaction guiding the correction of error-prone regions (Bai et al., 2021, Saukkoriipi et al., 10 Sep 2024).
• Multimodal and Multitask Systems: Some OCCs integrate clinical text information via LLMs or combine contouring with dose prediction in an MTL setup, enabling context-aware and efficient treatment planning (Oh et al., 2023, Kim et al., 27 Nov 2024).
• Promptable and Foundation Models: The most recent OCCs leverage foundation models with visual prompting (e.g., point, bounding-box input) to generate and propagate 3D segmentations, supporting human-in-the-loop validation and editing (Machado et al., 10 Oct 2024).

2. Algorithmic Methodologies

OCCs employ a spectrum of algorithms, which can be broadly categorized as follows:

System Stage	Algorithmic Approach	Example References
Detection	Feature extraction (SURF, FAST), seed-based region growing, CNNs	(Gangeh et al., 2017, Chen et al., 2019)
Initial Contour	3D U-Net, candidate nodule detection (RetinaUNet3D), cycleGANs for syn-MRI	(Dai et al., 2020, Astaraki et al., 2023, Luo et al., 19 Mar 2025)
Refinement	Level-set evolution (Chan–Vese), click-based interactive DL, SVMs	(Chen et al., 2019, Bai et al., 2021, Saukkoriipi et al., 10 Sep 2024)
Quality Control	CNN-GNN QA, Bayesian ordinal classification for output quality and uncertainty	(Henderson et al., 2022, Wang et al., 1 May 2025)
Enhancement	Multimodal alignment (LLMs w/ prompt tuning), multitask learning (MTL)	(Oh et al., 2023, Kim et al., 27 Nov 2024)

Notable algorithmic features include:

• Spatial feature augmentation to provide a prior relative to user-selected tumor centers (Gangeh et al., 2017).
• Dual pyramid networks and attention gating for feature fusion across imaging modalities (CBCT, sMRI) (Dai et al., 2020).
• Segmentation refinement with user interaction mapped as Gaussian-click images in an adjustable U-Net architecture (Bai et al., 2021, Saukkoriipi et al., 10 Sep 2024).
• Probabilistic contour generation to accurately model inter-observer variation (IOV) using IOV maps convolved with a spatial PSF (Osorio et al., 2022).
• Language Vision Models (LVMs) such as GPT-4V integrating structured prompts for vision-language fusion and false positive reduction during nodule identification (Luo et al., 19 Mar 2025).

3. Performance Metrics and Quantitative Evaluation

OCC systems are evaluated using standardized overlap and boundary metrics, including:

• Dice Similarity Coefficient (DSC): $D = \frac{2 |E \cap G|}{|E| + |G|}$, where E is the algorithmic (or OCC) estimate and G is the expert (ground-truth) contour (Gangeh et al., 2017, Kim et al., 27 Nov 2024).
  • Values reported: SURF-SVM achieves 76.7% (±15.2%) (Gangeh et al., 2017), 3D LSM methods reaching SI ≈ 85% (Chen et al., 2019), 3D U-Net-based OAR models ~0.887, GTVs ~0.790 (Astaraki et al., 2023), foundation models (ONCOPILOT) up to ~0.78 with interactive editing (Machado et al., 10 Oct 2024).
• Hausdorff Distance (HD95): The 95th percentile of boundary errors, sensitive to maximum deviations (Bai et al., 2021, Marin et al., 2021).
• Overlap Value (OV, Jaccard), Similarity Index (SI), Overlap Fraction (OF), Extra Fraction (EF): For more granular evaluation in clinical contexts (Chen et al., 2019).
• Task-specific Endpoints: Sensitivity, specificity, F1-score, and false discovery rate (FDR) for candidate-based detection and language-vision analysis; e.g., OCC achieving FDR = 0.511, ~1.714 FP/scan, F1 = 0.652 (Luo et al., 19 Mar 2025).
• Dose Prediction Metrics: Mean absolute DVH difference (DVH-MAE), 19.82% and 16.33% improvements for prostate and H&N datasets in integrated MTL models (Kim et al., 27 Nov 2024).

In clinical QA tasks for OART, Bayesian ordinal classification enabled >90% accuracy for auto-contour quality assessment, identifying high-quality contours confidently in >93% of cases (Wang et al., 1 May 2025).

4. Human-AI Collaboration and Interactive Workflows

A core principle of recent OCCs is maintaining a radiologist- or oncologist-in-the-loop paradigm:

• Interactive Editing: Users provide minimal feedback (e.g., clicks at error regions) that are translated into local constraint maps for DL-based correction. Most studies report 1–5 interactions to reach near ground-truth segmentation (e.g., DSC improves from 0.713 to 0.824 in five interactions; iteration latency ~20 ms) (Bai et al., 2021, Saukkoriipi et al., 10 Sep 2024).
• Uncertainty-Guided Review: Bayesian DLS models output contour proposals with quantifiable uncertainty; only slices or regions with elevated entropy/risk are flagged for focused review, minimizing exhaustive slice-by-slice editing (Chaves-de-Plaza et al., 2022).
• Redundancy-Aware Tools: Editing propagation tools apply corrections to multiple slices, reducing redundancy and keeping user effort proportional to actual risk (Chaves-de-Plaza et al., 2022).
• VR and Visual Analytics: Multi-modal VR platforms enhance spatial understanding, reducing mental burden and error rates—average DSC in VR-enabled modes ~82% vs. ~50% for 2D-only (Chen et al., 2022).

AI-derived uncertainty maps, consensus contours, and judgment aggregation protocols facilitate structured decision-making and support collaborative work, including independent expert review (Wahid et al., 2 Dec 2024).

5. Modeling Contouring Variation and Uncertainty

Explicit management of inter- and intra-observer variation is central to OCC methodology:

• Confidence Maps: DL models trained on multi-expert annotations output voxel-level probabilities reflecting consensus; these guide both automated proposals and highlight uncertainty zones (Marin et al., 2021).
• Realistic Simulation of Variability: Generation of random but plausible truth samples from measured IOV maps and PSF convolution enables probabilistic planning, awareness of geometric/dosimetric robustness, and realistic QA (Osorio et al., 2022).
• Noise Decomposition: Quantification of MSE into bias² and noise², including level and pattern noise, informs both model development and workflow integration (Wahid et al., 2 Dec 2024).

A plausible implication is that integrated uncertainty quantification can drive adaptive treatment margins and selective review, optimizing both safety and efficiency.

6. Clinical Integration, Automation, and Planning Support

Empirical studies report several clinical impacts:

• Automation of Routine Tasks: Most OCCs effect significant reductions in manual contouring time, e.g., threefold acceleration or more (auto-contouring in seconds to a minute vs. tens of minutes to hours; QA time for APT reduced from 30–60 min to ~5 min) (Astaraki et al., 2023, Chaves-de-Plaza et al., 2022).
• Consistency and Noise Reduction: AI models provide standardized contours, reducing inter-/intra-observer variation that is a major source of error in RT planning (Wahid et al., 2 Dec 2024).
• Integrated Planning: Multi-task learning unifies segmentation and dose prediction, cutting planning latency and propagating less error between sequential steps (Kim et al., 27 Nov 2024). Integrated frameworks demonstrated 19–20% improvements in DVH-MAE over sequential workflows.
• Expert Guidance in Resource-Limited Settings: The use of language vision integration (textual prompts aligned with imaging) allows for the incorporation of remote expert knowledge, sustaining standard of care in areas lacking subspecialty expertise (Luo et al., 19 Mar 2025).

7. Advances, Limitations, and Future Directions

OCC development is proceeding along multiple axes:

• Promptable Foundation Models: Broad pre-training (e.g., ONCOPILOT) with point/bounding-box prompting supports generalization and continuous improvement, with interactive editing outperforming in both speed and accuracy (Machado et al., 10 Oct 2024).
• Robust Multimodal Fusion: Incorporation of LLMs with learnable prompt tuning and multi-level alignment with image features enables context-aware segmentation that aligns with physician decision policies and real-world variability (Oh et al., 2023).
• Quality Control and Self-Supervised QA: Hybrid CNN-GNN and Bayesian ordinal classification enable robust and scalable QA with minimal ground-truth data (Henderson et al., 2022, Wang et al., 1 May 2025).
• Ongoing Challenges: Key obstacles include noise/artifact susceptibility in low-quality imaging (e.g., ultrasound), dependence on initial seed/center selection, residual dose distribution control trade-offs in MTL, performance on small or ambiguous lesions, and the need for prospective multi-center validation (Chen et al., 2019, Kim et al., 27 Nov 2024, Machado et al., 10 Oct 2024).

Anticipated future work focuses on improving uncertainty estimation, integrating real-time adaptation to user edits, further refining clinical text integration, and extending OCCs to multi-modal and multi-organ scenarios, with a continued emphasis on seamless user interaction and transparent decision support.

---

In summary, Oncology Contouring Copilots have evolved into feature-rich, clinically impactful systems exploiting advanced AI architectures, human-in-the-loop workflows, explicit uncertainty management, and standardized performance metrics. Their integration is reshaping radiotherapy planning by improving speed, reliability, and consistency while supporting clinician oversight at each critical juncture.