Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks (2009.11169v1)

Abstract: Traditional image-based survival prediction models rely on discriminative patch labeling which make those methods not scalable to extend to large datasets. Recent studies have shown Multiple Instance Learning (MIL) framework is useful for histopathological images when no annotations are available in classification task. Different to the current image-based survival models that limit to key patches or clusters derived from Whole Slide Images (WSIs), we propose Deep Attention Multiple Instance Survival Learning (DeepAttnMISL) by introducing both siamese MI-FCN and attention-based MIL pooling to efficiently learn imaging features from the WSI and then aggregate WSI-level information to patient-level. Attention-based aggregation is more flexible and adaptive than aggregation techniques in recent survival models. We evaluated our methods on two large cancer whole slide images datasets and our results suggest that the proposed approach is more effective and suitable for large datasets and has better interpretability in locating important patterns and features that contribute to accurate cancer survival predictions. The proposed framework can also be used to assess individual patient's risk and thus assisting in delivering personalized medicine. Codes are available at https://github.com/uta-smile/DeepAttnMISL_MEDIA.

• The paper presents a novel DeepAttnMISL framework that leverages deep attention within a multiple instance learning approach to predict cancer survival outcomes.
• It clusters image patches into phenotypes using a Siamese MI-FCN and an attention pooling layer, enhancing interpretability without exhaustive annotations.
• Empirical results on lung and colorectal cancer datasets demonstrate superior C-index and AUC performance compared to traditional methods.

Survival Prediction from Whole Slide Images using Deep Attention Multiple Instance Learning

The paper "Whole Slide Images based Cancer Survival Prediction using Attention Guided Deep Multiple Instance Learning Networks" proposes a novel framework, DeepAttnMISL, to predict cancer survival outcomes from Whole Slide Images (WSIs) using a combination of deep learning techniques, specifically leveraging the Multiple Instance Learning (MIL) approach. This approach is significant due to its ability to process data in a weakly supervised manner, addressing the challenges posed by the massive scale and complexity of WSIs without requiring pixel-level annotations.

In the landscape of histopathological image analysis, traditional methods have heavily relied on examining discriminative patches within images, which are often manually annotated and selected by expert pathologists. This process is labor-intensive and not scalable to larger datasets due to the heterogeneity and gigapixel resolution of WSIs. DeepAttnMISL distinguishes itself by employing MIL, specifically utilizing phenotype clustering to group image patches into distinct types based on morphology and then analyzing this data with a Siamese network architecture that efficiently learns meaningful representations from the image data.

DeepAttnMISL introduces several innovations. Foremost, it employs a deep attention mechanism within an MIL framework, which enhances the interpretability and effectiveness of the learning process by facilitating the identification of phenotypic clusters that significantly influence patient outcomes. The model's flexibility and scalability are demonstrated through its evaluation on large datasets from lung and colorectal cancer cohorts, showing superior performance compared to traditional techniques and demonstrating capability without the burden of exhaustive manual annotation.

Methodologically, the paper outlines that the WSI data is processed by first cutting it into patches, followed by feature extraction using pre-trained models, thereby reducing computational burdens associated with training from scratch. Patches are then clustered into phenotypes on a per-patient basis. The DeepAttnMISL model uses a Siamese MI-FCN to process each phenotypic cluster, feeding into an attention-based pooling mechanism that aggregates them into patient-level predictions. The developments presented in this work include the implementation of an attention-pooling layer, which determines the importance of each phenotype cluster, effectively facilitating more informed predictions of patient survival risks.

Empirical evaluations, shown via C-index and AUC metrics, underline the efficacy of DeepAttnMISL. The model outperforms other state-of-the-art methods, including those based on deep convolutional networks applied at the tile level followed by separate learning stages such as WSISA. Furthermore, the attention mechanism grants superior interpretability, revealing which phenotypical patterns are most indicative of prognosis, thereby aiding in better individualized treatment planning.

The paper suggests future extensions of DeepAttnMISL may include applications to other cancer types beyond colorectal and lung, given the generalized framework of the methodology. As WSIs continue to grow in utility within the clinical setting, approaches like DeepAttnMISL that can deliver accurate, interpretable predictions at scale without exhaustive annotations are poised to make significant contributions to personalized medicine and treatment decision-making in oncology.