Weakly Supervised AI for Efficient Analysis of 3D Pathology Samples (2307.14907v1)

Abstract: Human tissue and its constituent cells form a microenvironment that is fundamentally three-dimensional (3D). However, the standard-of-care in pathologic diagnosis involves selecting a few two-dimensional (2D) sections for microscopic evaluation, risking sampling bias and misdiagnosis. Diverse methods for capturing 3D tissue morphologies have been developed, but they have yet had little translation to clinical practice; manual and computational evaluations of such large 3D data have so far been impractical and/or unable to provide patient-level clinical insights. Here we present Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA), a deep-learning-based platform for processing 3D tissue images from diverse imaging modalities and predicting patient outcomes. Archived prostate cancer specimens were imaged with open-top light-sheet microscopy or microcomputed tomography and the resulting 3D datasets were used to train risk-stratification networks based on 5-year biochemical recurrence outcomes via MAMBA. With the 3D block-based approach, MAMBA achieves an area under the receiver operating characteristic curve (AUC) of 0.86 and 0.74, superior to 2D traditional single-slice-based prognostication (AUC of 0.79 and 0.57), suggesting superior prognostication with 3D morphological features. Further analyses reveal that the incorporation of greater tissue volume improves prognostic performance and mitigates risk prediction variability from sampling bias, suggesting the value of capturing larger extents of heterogeneous 3D morphology. With the rapid growth and adoption of 3D spatial biology and pathology techniques by researchers and clinicians, MAMBA provides a general and efficient framework for 3D weakly supervised learning for clinical decision support and can help to reveal novel 3D morphological biomarkers for prognosis and therapeutic response.

• The paper presents the MAMBA framework, which uses weak supervision to analyze 3D pathology images without extensive manual annotations.
• It employs a multi-step process of segmentation, feature extraction, and attention-based aggregation to predict patient risk.
• The framework outperforms traditional 2D evaluations by achieving an AUC of 0.86 and providing insights into tissue heterogeneity.

Weakly Supervised AI for Analyzing 3D Pathology Samples

The paper presents a pioneering approach to the utilization of weakly supervised AI for the analysis of 3D pathology images, specifically through the development of the Modality-Agnostic Multiple instance learning for volumetric Block Analysis (MAMBA) framework. MAMBA represents a deep learning-based computational tool designed to process three-dimensional tissue images obtained from diverse imaging modalities, enhancing the prognostication of patient outcomes in a clinical context.

Motivation and Background

Traditional pathology often involves the analysis of two-dimensional (2D) sections from three-dimensional (3D) tissue samples. This practice, although well-established, inherently carries the risk of sampling bias and potential misdiagnosis due to the limited scope of morphological information captured in a single slice. As there is increasing recognition of the limitations of 2D pathology, efforts have intensified in transitioning to 3D pathology. However, the adoption of 3D approaches in clinical processes has been hampered due to the manual impracticality of reviewing large volumetric data and the lack of computational platforms to efficiently process and interpret these data.

MAMBA Framework

MAMBA introduces a novel approach that emphasizes modality-agnosticism and efficiency, focusing on weakly supervised learning paradigms to analyze 3D pathology images without requiring extensive manual annotation. The platform employs a deep learning-based multiple instance learning (MIL) methodology, which comprises three primary steps:

1. Preprocessing: The process involves segmentation of the 3D volumes into a series of planes or cuboids, which are further divided into smaller patches for processing.
2. Feature Extraction: Each patch undergoes encoding via a combination of a pre-trained feature extraction network and a shallow network, resulting in a compact representation of the morphology.
3. Aggregation: Attention-based modules weigh and aggregate the patch-wise features to produce a composite feature vector, forming the basis for patient-level risk prediction.

The platform's integration of attention-based aggregation allows the system to identify critical morphological regions contributing to the risk predictions autonomously, thereby mitigating the need for pixel-level annotations typically necessary in similar computational models.

Evaluation and Results

The framework was evaluated on different datasets, including simulated 3D phantom data and two real-world prostate cancer cohorts processed via open-top light-sheet microscopy (OTLS) and microcomputed tomography (microCT). Notable findings from these evaluations include:

• Performance Superiority: MAMBA demonstrated superior performance in prognostication compared to traditional 2D slice-based evaluations. For instance, the 3D block-based approach achieved an AUC of 0.86, significantly exceeding the 2D method's AUC of 0.79.
• Weak Supervision Benefit: The incorporation of weakly supervised learning allowed the processing of vast 3D data with minimal manual annotations, capturing intricate 3D morphological features that offer prognostic insights superior to those derived from 2D analyses.
• Insight into Tissue Heterogeneity: MAMBA's capability to analyze entire tissue volumes reduced variability in risk predictions emerging from spatial heterogeneity.

Implications and Future Directions

The MAMBA framework marks a significant advancement in computational pathology, highlighting the transition towards comprehensive 3D tissue analysis in clinical settings. The platform's modality-agnostic design aligns with the burgeoning diversity of 3D imaging technologies, potentially facilitating broader clinical adaptation. Furthermore, MAMBA's ability to process volumes more efficiently than 2D approaches could play a crucial role in handling increasingly larger datasets with advancing imaging modalities.

Future research directions could explore the integration of MAMBA with additional lens designs or molecular imaging modalities, enhancing its diagnostic capabilities. As 3D pathology gains traction, the development of enhanced deep learning techniques optimized for volumetric data analysis is expected to reveal novel morphological biomarkers, further advancing precision medicine.