Integrating multiscale topology in digital pathology with pyramidal graph convolutional networks (2403.15068v1)
Abstract: Graph convolutional networks (GCNs) have emerged as a powerful alternative to multiple instance learning with convolutional neural networks in digital pathology, offering superior handling of structural information across various spatial ranges - a crucial aspect of learning from gigapixel H&E-stained whole slide images (WSI). However, graph message-passing algorithms often suffer from oversmoothing when aggregating a large neighborhood. Hence, effective modeling of multi-range interactions relies on the careful construction of the graph. Our proposed multi-scale GCN (MS-GCN) tackles this issue by leveraging information across multiple magnification levels in WSIs. MS-GCN enables the simultaneous modeling of long-range structural dependencies at lower magnifications and high-resolution cellular details at higher magnifications, akin to analysis pipelines usually conducted by pathologists. The architecture's unique configuration allows for the concurrent modeling of structural patterns at lower magnifications and detailed cellular features at higher ones, while also quantifying the contribution of each magnification level to the prediction. Through testing on different datasets, MS-GCN demonstrates superior performance over existing single-magnification GCN methods. The enhancement in performance and interpretability afforded by our method holds promise for advancing computational pathology models, especially in tasks requiring extensive spatial context.
- “A framework for multiple-instance learning,” Advances in neural information processing systems, vol. 10, 1997.
- “Multiple instance learning for digital pathology: A review on the state-of-the-art, limitations & future potential,” arXiv preprint arXiv:2206.04425, 2022.
- “Histocartography: A toolkit for graph analytics in digital pathology,” in MICCAI Workshop on Computational Pathology. PMLR, 2021, pp. 117–128.
- “Slidegraph+: whole slide image level graphs to predict her2 status in breast cancer,” Medical Image Analysis, vol. 80, pp. 102486, 2022.
- “Whole slide images are 2d point clouds: Context-aware survival prediction using patch-based graph convolutional networks,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2021, pp. 339–349.
- Thomas de Bel et al., “Automatic segmentation of histopathological slides of renal tissue using deep learning,” in Medical Imaging 2018: Digital Pathology. SPIE, 2018, vol. 10581, pp. 285–290.
- Joshua J Levy et al., “Pathflowai: a high-throughput workflow for preprocessing, deep learning and interpretation in digital pathology,” in Pacific symposium on biocomputing, 2020, pp. 403–414.
- Mart Van Rijthoven et al., “Hooknet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images,” Medical image analysis, vol. 68, pp. 101890, 2021.
- “A multi-resolution model for histopathology image classification and localization with multiple instance learning,” Computers in biology and medicine, vol. 131, pp. 104253, 2021.
- “Multi-scale relational graph convolutional network for multiple instance learning in histopathology images,” arXiv preprint arXiv:2212.08781, 2022.
- “Deepgcns: Can gcns go as deep as cnns?,” in Proceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 9267–9276.
- “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907, 2016.
- “Deepergcn: All you need to train deeper gcns,” arXiv preprint arXiv:2006.07739, 2020.
- Maximilian Ilse et al., “Attention-based deep multiple instance learning,” in International conference on machine learning. PMLR, 2018, pp. 2127–2136.
- Wouter Bulten et al., “Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge,” Nature medicine, vol. 28, no. 1, pp. 154–163, 2022.