Multi-modal AI for comprehensive breast cancer prognostication (2410.21256v2)

Abstract: Treatment selection in breast cancer is guided by molecular subtypes and clinical characteristics. However, current tools including genomic assays lack the accuracy required for optimal clinical decision-making. We developed a novel AI-based approach that integrates digital pathology images with clinical data, providing a more robust and effective method for predicting the risk of cancer recurrence in breast cancer patients. Specifically, we utilized a vision transformer pan-cancer foundation model trained with self-supervised learning to extract features from digitized H&E-stained slides. These features were integrated with clinical data to form a multi-modal AI test predicting cancer recurrence and death. The test was developed and evaluated using data from a total of 8,161 female breast cancer patients across 15 cohorts originating from seven countries. Of these, 3,502 patients from five cohorts were used exclusively for evaluation, while the remaining patients were used for training. Our test accurately predicted our primary endpoint, disease-free interval, in the five evaluation cohorts (C-index: 0.71 [0.68-0.75], HR: 3.63 [3.02-4.37, p<0.001]). In a direct comparison (n=858), the AI test was more accurate than Oncotype DX, the standard-of-care 21-gene assay, achieving a C-index of 0.67 [0.61-0.74] versus 0.61 [0.49-0.73], respectively. Additionally, the AI test added independent prognostic information to Oncotype DX in a multivariate analysis (HR: 3.11 [1.91-5.09, p<0.001)]). The test demonstrated robust accuracy across major molecular breast cancer subtypes, including TNBC (C-index: 0.71 [0.62-0.81], HR: 3.81 [2.35-6.17, p=0.02]), where no diagnostic tools are currently recommended by clinical guidelines. These results suggest that our AI test improves upon the accuracy of existing prognostic tests, while being applicable to a wider range of patients.

• The paper introduces an AI test that combines vision transformer-based digital pathology with clinical data to enhance breast cancer prognostication.
• The methodology uses a self-supervised model (Kestrel) trained on 400 million image patches, achieving a C-index of 0.71 and HR of 3.63.
• The results highlight that integrating multi-modal AI into clinical workflows can personalize treatment strategies and outperform standard genomic assays.

Multi-modal AI for Comprehensive Breast Cancer Prognostication

This paper presents a significant advancement in breast cancer prognostication through the development and evaluation of a multi-modal AI test. The research addresses the limitations of current breast cancer risk assessment methods, which primarily rely on genomic assays and clinicopathological parameters. These existing methods are often accurate only for specific subtypes and present limited clinical utility due to the requirements for physical tissue processing and potential tissue depletion.

Methodology and Approach

The paper introduces a novel AI test that integrates digital pathology and clinical data to predict breast cancer recurrence and mortality. The test utilizes a vision transformer-based foundation model, named Kestrel, which employs self-supervised learning. Trained on a dataset comprising 400 million digital pathology image patches, Kestrel autonomously learns features from digitized H&E-stained slides. These extracted features are combined with clinical data to create a multi-modal prognostic model.

Dataset and Evaluation

The AI test was rigorously evaluated using data from 8,161 breast cancer patients across 15 cohorts from seven countries. For evaluation, 3,502 patients across five cohorts were used, while the remainder contributed to training. The AI model demonstrated a superior predictive capability with a C-index of 0.71 and a hazard ratio (HR) of 3.63, outperforming Oncotype DX — a standard 21-gene assay — which exhibited a C-index of 0.61 in direct comparison.

Results and Findings

Across all major breast cancer subtypes, the AI test showed robust prognostic accuracy. In triple-negative breast cancer (TNBC), which lacks clinically recommended diagnostic tools, the model achieved a C-index of 0.71 with an HR of 3.81. These strong numerical outcomes underline the model's potential to not only improve accuracy but also enhance the applicability and accessibility of treatment selection tools.

Implications and Future Directions

The implications of this research are multifaceted. Practically, the multi-modal AI test promises to enhance personalized treatment strategies by providing more comprehensive risk assessments that are not limited to certain subtypes or demographic groups. Theoretically, the paper demonstrates the effectiveness of combining self-supervised learning in digital pathology with multi-modal AI frameworks.

Looking ahead, the success of this model opens avenues for further research into AI's application in broader cancer prognostication tasks. Prospective studies incorporating randomized data could explore the potential predictive capabilities of the AI test concerning treatment outcomes. Additionally, future developments might focus on integrating such AI models into clinical workflows, thereby improving timeliness and reducing costs compared to genomic assays, while preserving valuable biological samples.

In conclusion, this paper presents a sophisticated and promising tool for the prognosis of breast cancer, highlighting an important step toward improving personalized medicine through AI innovations.