Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CAMIL: Context-Aware Multiple Instance Learning for Cancer Detection and Subtyping in Whole Slide Images (2305.05314v3)

Published 9 May 2023 in cs.CV

Abstract: The visual examination of tissue biopsy sections is fundamental for cancer diagnosis, with pathologists analyzing sections at multiple magnifications to discern tumor cells and their subtypes. However, existing attention-based multiple instance learning (MIL) models used for analyzing Whole Slide Images (WSIs) in cancer diagnostics often overlook the contextual information of tumor and neighboring tiles, leading to misclassifications. To address this, we propose the Context-Aware Multiple Instance Learning (CAMIL) architecture. CAMIL incorporates neighbor-constrained attention to consider dependencies among tiles within a WSI and integrates contextual constraints as prior knowledge into the MIL model. We evaluated CAMIL on subtyping non-small cell lung cancer (TCGA-NSCLC) and detecting lymph node (CAMELYON16 and CAMELYON17) metastasis, achieving test AUCs of 97.5\%, 95.9\%, and 88.1\%, respectively, outperforming other state-of-the-art methods. Additionally, CAMIL enhances model interpretability by identifying regions of high diagnostic value.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Representation learning of histopathology images using graph neural networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2020, Seattle, WA, USA, June 14-19, 2020, pages 4254–4261. Computer Vision Foundation / IEEE, 2020. doi: 10.1109/CVPRW50498.2020.00502.
  2. Predicting cancer with a recurrent visual attention model for histopathology images. In Alejandro F. Frangi, Julia A. Schnabel, Christos Davatzikos, Carlos Alberola-López, and Gabor Fichtinger, editors, Medical Image Computing and Computer Assisted Intervention - MICCAI 2018 - 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part II, volume 11071 of Lecture Notes in Computer Science, pages 129–137. Springer, 2018. doi: 10.1007/978-3-030-00934-2_15.
  3. Spectral clustering with graph neural networks for graph pooling. 2020.
  4. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, pages 1–9, 2019.
  5. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR, 2020.
  6. Computational Image Analysis Identifies Histopathological Image Features Associated With Somatic Mutations and Patient Survival in Gastric Adenocarcinoma. Frontiers in Oncology, 11, 2021. ISSN 2234-943X.
  7. Classification and disease localization in histopathology using only global labels: A weakly-supervised approach. CoRR, abs/1802.02212, 2018.
  8. Multiple instance learning of deep convolutional neural networks for breast histopathology whole slide classification. In 15th IEEE International Symposium on Biomedical Imaging, ISBI 2018, Washington, DC, USA, April 4-7, 2018, pages 578–581. IEEE, 2018. doi: 10.1109/ISBI.2018.8363642.
  9. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. doi: 10.1109/CVPR.2009.5206848.
  10. Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA, 318(22):2199–2210, December 2017. ISSN 0098-7484. doi: 10.1001/jama.2017.14585. URL https://doi.org/10.1001/jama.2017.14585.
  11. Deep residual learning for image recognition, 2015.
  12. Patch-based convolutional neural network for whole slide tissue image classification. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 2424–2433. IEEE Computer Society, 2016. doi: 10.1109/CVPR.2016.266.
  13. Attention-based deep multiple instance learning. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2132–2141, 2018.
  14. Colour deconvolution: stain unmixing in histological imaging. Bioinformatics, 09 2020. ISSN 1367-4803. doi: 10.1093/bioinformatics/btaa847. btaa847.
  15. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. CoRR, abs/2011.08939, 2020. URL https://arxiv.org/abs/2011.08939.
  16. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 14318–14328. Computer Vision Foundation / IEEE, 2021a. doi: 10.1109/CVPR46437.2021.01409. URL https://openaccess.thecvf.com/content/CVPR2021/html/Li_Dual-Stream_Multiple_Instance_Learning_Network_for_Whole_Slide_Image_Classification_CVPR_2021_paper.html.
  17. A multi-resolution model for histopathology image classification and localization with multiple instance learning. Comput. Biol. Medicine, 131:104253, 2021b. doi: 10.1016/j.compbiomed.2021.104253.
  18. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5(6):555–570, 2021.
  19. A novel multiple-instance learning-based approach to computer-aided detection of tuberculosis on chest x-rays. IEEE Trans. Medical Imaging, 34(1):179–192, 2015. doi: 10.1109/TMI.2014.2350539.
  20. Artificial intelligence in computational pathology - challenges and future directions. Digit. Signal Process., 119:103196, 2021. doi: 10.1016/j.dsp.2021.103196.
  21. Multiple-instance learning for anomaly detection in digital mammography. IEEE Trans. Medical Imaging, 35(7):1604–1614, 2016. doi: 10.1109/TMI.2016.2521442.
  22. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pages 2136–2147, 2021a.
  23. TransMIL: Transformer based correlated multiple instance learning for whole slide image classification. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, 2021b. URL https://openreview.net/forum?id=LKUfuWxajHc.
  24. Cluster-to-conquer: A framework for end-to-end multi-instance learning for whole slide image classification. In Mattias P. Heinrich, Qi Dou, Marleen de Bruijne, Jan Lellmann, Alexander Schlaefer, and Floris Ernst, editors, Medical Imaging with Deep Learning, 7-9 July 2021, Lübeck, Germany, volume 143 of Proceedings of Machine Learning Research, pages 682–698. PMLR, 2021.
  25. Deep neural network models for computational histopathology: A survey. CoRR, abs/1912.12378, 2019.
  26. Neural image compression for gigapixel histopathology image analysis. IEEE Trans. Pattern Anal. Mach. Intell., 43(2):567–578, 2021. doi: 10.1109/TPAMI.2019.2936841.
  27. Multiple instance learning for classification of dementia in brain MRI. Medical Image Anal., 18(5):808–818, 2014. doi: 10.1016/j.media.2014.04.006.
  28. Multiple instance learning with graph neural networks. CoRR, abs/1906.04881, 2019.
  29. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998–6008, 2017.
  30. Predicting gastric cancer outcome from resected lymph node histopathology images using deep learning. Nature Communications, 12(1):1637, March 2021. ISSN 2041-1723. doi: 10.1038/s41467-021-21674-7. Number: 1 Publisher: Nature Publishing Group.
  31. Revisiting multiple instance neural networks. Pattern Recognit., 74:15–24, 2018. doi: 10.1016/j.patcog.2017.08.026.
  32. Beyond classification: Whole slide tissue histopathology analysis by end-to-end part learning. In Tal Arbel, Ismail Ben Ayed, Marleen de Bruijne, Maxime Descoteaux, Hervé Lombaert, and Christopher Pal, editors, International Conference on Medical Imaging with Deep Learning, MIDL 2020, 6-8 July 2020, Montréal, QC, Canada, volume 121 of Proceedings of Machine Learning Research, pages 843–856, 2020.
  33. Nyströmformer: A nyström-based algorithm for approximating self-attention. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 14138–14148. AAAI Press, 2021. URL https://ojs.aaai.org/index.php/AAAI/article/view/17664.
  34. CAMEL: A weakly supervised learning framework for histopathology image segmentation. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 10681–10690, 2019. doi: 10.1109/ICCV.2019.01078.
  35. Weakly supervised histopathology cancer image segmentation and classification. Medical Image Anal., 18(3):591–604, 2014. doi: 10.1016/j.media.2014.01.010.
  36. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Anal., 65:101789, 2020. doi: 10.1016/j.media.2020.101789.
  37. DTFD-MIL: double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 18780–18790, 2022. doi: 10.1109/CVPR52688.2022.01824.
  38. A joint spatial and magnification based attention framework for large scale histopathology classification. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021, virtual, June 19-25, 2021, pages 3776–3784. Computer Vision Foundation / IEEE, 2021. doi: 10.1109/CVPRW53098.2021.00418.
  39. Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 4836–4845. Computer Vision Foundation / IEEE. doi: 10.1109/CVPR42600.2020.00489.
  40. A graph-transformer for whole slide image classification. IEEE Trans. Medical Imaging, 41(11):3003–3015, 2022. doi: 10.1109/TMI.2022.3176598. URL https://doi.org/10.1109/TMI.2022.3176598.
  41. Histopathology classification and localization of colorectal cancer using global labels by weakly supervised deep learning. Comput. Medical Imaging Graph., 88:101861, 2021. doi: 10.1016/j.compmedimag.2021.101861.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Olga Fourkioti (1 paper)
  2. Matt De Vries (1 paper)
  3. Chris Bakal (2 papers)
  4. Chen Jin (18 papers)
  5. Daniel C. Alexander (82 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now