Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multiple Instance Learning for Digital Pathology: A Review on the State-of-the-Art, Limitations & Future Potential (2206.04425v2)

Published 9 Jun 2022 in cs.CV

Abstract: Digital whole slides images contain an enormous amount of information providing a strong motivation for the development of automated image analysis tools. Particularly deep neural networks show high potential with respect to various tasks in the field of digital pathology. However, a limitation is given by the fact that typical deep learning algorithms require (manual) annotations in addition to the large amounts of image data, to enable effective training. Multiple instance learning exhibits a powerful tool for learning deep neural networks in a scenario without fully annotated data. These methods are particularly effective in this domain, due to the fact that labels for a complete whole slide image are often captured routinely, whereas labels for patches, regions or pixels are not. This potential already resulted in a considerable number of publications, with the majority published in the last three years. Besides the availability of data and a high motivation from the medical perspective, the availability of powerful graphics processing units exhibits an accelerator in this field. In this paper, we provide an overview of widely and effectively used concepts of used deep multiple instance learning approaches, recent advances and also critically discuss remaining challenges and future potential.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. From whence will they come? a perspective on the acute shortage of pathologists in biomedical research. Journal of Veterinary Diagnostic Investigation 19, 455–456.
  2. Topology aware fully convolutional networks for histology gland segmentation, in: Proceedings of the Conference on Medical Image Compututing and Computer Assisted Interventions, Springer. pp. 460–468.
  3. Bringing open data to whole slide imaging, in: Proceedings of the European Congress on Digital Pathology, pp. 3–10.
  4. End-to-end Multiple Instance Learning for Whole-Slide Cytopathology of Urothelial Carcinoma, in: Proceedings of the International Conference on Medical Image Computing and Computer Assisted Interventions, pp. 57–68.
  5. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine 25, 1301–1309.
  6. Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition 77, 329–353.
  7. A simple framework for contrastive learning of visual representations, in: Proceedings of the International Conference on Machine Learning, pp. 1597–1607.
  8. Interobserver variability in upfront dichotomous histopathological assessment of ductal carcinoma in situ of the breast: the DCISion study. Modern Pathology 33, 354–366.
  9. Imagenet: A large-scale hierarchical image database, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255.
  10. An image is worth 16x16 words: Transformers for image recognition at scale, in: Proceedings of the International Conference on Learning Representations (ICLR).
  11. U-Net: deep learning for cell counting, detection, and morphometry. Nature Methods 16, 67–70.
  12. A comprehensive review on multiple instance learning. Electronics 12, 4323.
  13. Structured state space models for multiple instance learning in digital pathology, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 594–604.
  14. Improving the segmentation of anatomical structures in chest radiographs using u-net with an imagenet pre-trained encoder, in: Proceedings of the International MICCAI Workshop on Reconstruction and Analysis of Moving Body Organs, pp. 159–168.
  15. Mixup-mil: Novel data augmentation for multiple instance learning and a study on thyroid cancer diagnosis, in: Proceedings of the Conference on Medical Image Computing and Computer Aided Interventions (MICCAI).
  16. Frozen-to-paraffin: Categorization of histological frozen sections by the aid of paraffin sections and generative adversarial networks, in: Proceedings of the International MICCAI Workshop on Simulation and Synthesis in Medical Imaging, pp. 99–109.
  17. A semi-supervised multi-task learning framework for cancer classification with weak annotation in whole-slide images. Medical Image Analysis 83, 102652.
  18. A survey on vision transformer. IEEE transactions on pattern analysis and machine intelligence 45, 87–110.
  19. Integrating digital pathology into clinical practice. Mod. Pathol. 35, 152–164.
  20. Multi-scale domain-adversarial multiple-instance CNN for cancer subtype classification with unannotated histopathological images, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3852–3861.
  21. Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778.
  22. Deep interactive learning-based ovarian cancer segmentation of h&e-stained whole slide images to study morphological patterns of brca mutation. Journal of Pathology Informatics 14, 100160.
  23. Patch-based convolutional neural network for whole slide tissue image classification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2424–2433.
  24. Tailored for Real-World: A Whole Slide Image Classification System Validated on Uncurated Multi-Site Data Emulating the Prospective Pathology Workload. Scientific Reports 10, 3217.
  25. Attention-based deep multiple instance learning, in: Proceedings of the International Conference on Machine Learning, pp. 2127–2136.
  26. Deep multiple instance learning for digital histopathology, in: Handbook of Medical Image Computing and Computer Assisted Intervention, pp. 521–546.
  27. Transformers in vision: A survey. ACM computing surveys (CSUR) 54, 1–41.
  28. Autoencoding beyond pixels using a learned similarity metric, in: Proceedings of the International Conference on Machine Learning, pp. 1558–1566.
  29. Weakly supervised multiple instance learning histopathological tumor segmentation, in: Proceedings of the International Conference on Medical Image Computing and Computer Assisted Interventions. arXiv:2004.05024.
  30. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 14318–14328.
  31. Dt-mil: deformable transformer for multi-instance learning on histopathological image, in: Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VIII 24, pp. 206–216.
  32. A novel multiple instance learning framework for COVID-19 severity assessment via data augmentation and self-supervised learning. Medical Image Analysis 69, 101978.
  33. Interventional bag multi-instance learning on whole-slide pathological images, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19830–19839.
  34. Key instance detection in multi-instance learning, in: Proceedings of the Asian Conference on Machine Learning, pp. 253–268.
  35. Multiple instance learning via iterative self-paced supervised contrastive learning, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3355–3365.
  36. MetaCon: Meta contrastive learning for microsatellite instability detection, in: Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), pp. 267–276.
  37. AI-based pathology predicts origins for cancers of unknown primary. Nature 594, 106–110.
  38. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5, 555–570.
  39. A method for normalizing histology slides for quantitative analysis, in: Proceedings of the Inernational Symposium on Biomedical Imaging, pp. 1107–1110.
  40. Addressing the shortage of pathologists in Africa: Creation of a MMed Programme in Pathology in Zambia. African Journal of Laboratory Medicine 9, 1–7.
  41. Accounting for dependencies in deep learning based multiple instance learning for whole slide imaging, in: Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), pp. 329–338.
  42. Obtaining spatially resolved tumor purity maps using deep multiple instance learning in a pan-cancer study. Patterns 3, 100399.
  43. Studying The Effect of MIL Pooling Filters on MIL Tasks. arXiv preprint arXiv:2006.01561 , 1–16URL: http://arxiv.org/abs/2006.01561, arXiv:2006.01561.
  44. Towards label-efficient automatic diagnosis and analysis: a comprehensive survey of advanced deep learning-based weakly-supervised, semi-supervised and self-supervised techniques in histopathological image analysis. Physics in Medicine & Biology .
  45. Iib-mil: Integrated instance-level and bag-level multiple instances learning with label disambiguation for pathological image analysis, in: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer. pp. 560–569.
  46. Deep weakly-supervised learning methods for classification and localization in histology images: a survey. arXiv preprint arXiv:1909.03354 .
  47. Kernel self-attention for weakly-supervised image classification using deep multiple instance learning, in: Proceedings of the IEEE Winter Conference on Applications of Computer Vision, pp. 1721–1730.
  48. Survival prediction in triple negative breast cancer using multiple instance learning of histopathological images. Scientific Reports 12.
  49. Staingan: Stain style transfer for digital histological images, in: Proceedings of the Inernational Symposium on Biomedical Imaging, pp. 953–956.
  50. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. Advances in Neural Information Processing Systems (NIPS) 34.
  51. Cluster-to-conquer: A framework for end-to-end multi-instance learning for whole slide image classification, in: Proceedings of the International Conference on Medical Imaging with Deep Learning, pp. 682–698.
  52. Explainable deep learning models in medical image analysis. Journal of Imaging 6, 52.
  53. Scorenet: Learning non-uniform attention and augmentation for transformer-based histopathological image classification, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 6170–6179.
  54. Attention2majority: Weak multiple instance learning for regenerative kidney grading on whole slide images. Medical Image Analysis 79, 102462.
  55. Multiple instance learning for histopathological breast cancer image classification. Expert Systems with Applications 117, 103–111.
  56. Weakly supervised learning for classification of lung cytological images using attention-based multiple instance learning. Scientific Reports 11.
  57. Differentiable zooming for multiple instance learning on whole-slide images, in: Proceedings of the European Conference on Computer Vision (ECCV), pp. 699–715.
  58. Artificial intelligence and digital pathology: challenges and opportunities. Journal of Pathololgy Informatics 9.
  59. Evaluation of Multi-Scale Multiple Instance Learning to Improve Thyroid Cancer Classification, in: Proceedings of the International Conference on Image Processing Theory, Tools, and Applications.
  60. Generative Adversarial Networks in Digital Pathology: A Survey on Trends and Future Potential. Patterns 1, 100089. doi:10.1016/j.patter.2020.100089.
  61. Interobserver variability in the histopathologic assessment of extrathyroidal extension of well differentiated thyroid carcinoma supports the new american joint committee on cancer eighth edition criteria for tumor staging. Thyroid 29, 619–624.
  62. Interobserver variability in ductal carcinoma in situ of the breast. American Journal on Clinical Pathology 154, 596–609.
  63. Iteratively coupled multiple instance learning from instance to bag classifier for whole slide image classification, in: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI).
  64. Label cleaning multiple instance learning: Refining coarse annotations on single whole-slide images. IEEE Transactions on Medical Imaging 41, 3952–3968.
  65. Recent advances in variational autoencoders with representation learning for biomedical informatics: A survey. IEEE Access 9, 4939–4956.
  66. Pre-training on grayscale imagenet improves medical image classification, in: In Proceedings of the European Conference on Computer Vision, p. 0.
  67. Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. Medical Image Analysis 65, 101789.
  68. Competitive probabilistic neural network. Integrated Computer-Aided Engineering 24, 105–118.
  69. Self-attention generative adversarial networks, in: Proceedings of the International Conference on Machine Learning, pp. 7354–7363.
  70. Dtfd-mil: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18802–18812.
  71. Predicting lymph node metastasis using histopathological images based on multiple instance learning with deep graph convolution, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4837–4846.
Citations (42)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now