Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
175 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Multiple Instance Learning Framework with Masked Hard Instance Mining for Whole Slide Image Classification (2307.15254v3)

Published 28 Jul 2023 in cs.CV and cs.AI

Abstract: The whole slide image (WSI) classification is often formulated as a multiple instance learning (MIL) problem. Since the positive tissue is only a small fraction of the gigapixel WSI, existing MIL methods intuitively focus on identifying salient instances via attention mechanisms. However, this leads to a bias towards easy-to-classify instances while neglecting hard-to-classify instances. Some literature has revealed that hard examples are beneficial for modeling a discriminative boundary accurately. By applying such an idea at the instance level, we elaborate a novel MIL framework with masked hard instance mining (MHIM-MIL), which uses a Siamese structure (Teacher-Student) with a consistency constraint to explore the potential hard instances. With several instance masking strategies based on attention scores, MHIM-MIL employs a momentum teacher to implicitly mine hard instances for training the student model, which can be any attention-based MIL model. This counter-intuitive strategy essentially enables the student to learn a better discriminating boundary. Moreover, the student is used to update the teacher with an exponential moving average (EMA), which in turn identifies new hard instances for subsequent training iterations and stabilizes the optimization. Experimental results on the CAMELYON-16 and TCGA Lung Cancer datasets demonstrate that MHIM-MIL outperforms other latest methods in terms of performance and training cost. The code is available at: https://github.com/DearCaat/MHIM-MIL.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. An improved deep learning architecture for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3908–3916, 2015.
  2. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA, 318(22):2199–2210, 2017.
  3. Signature verification using a” siamese” time delay neural network. Advances in neural information processing systems, 6, 1993.
  4. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine, 25(8):1301–1309, 2019.
  5. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021.
  6. Scaling vision transformers to gigapixel images via hierarchical self-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16144–16155, 2022.
  7. Beyond triplet loss: a deep quadruplet network for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 403–412, 2017.
  8. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750–15758, 2021.
  9. Dual attention multiple instance learning with unsupervised complementary loss for covid-19 screening. Medical Image Analysis, 72:102105, 2021.
  10. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  11. Solving the multiple instance problem with axis-parallel rectangles. Artificial intelligence, 89(1-2):31–71, 1997.
  12. Solving the multiple instance problem with axis-parallel rectangles. Artificial Intelligence, 89(1-2):31–71, 1997.
  13. Class rectification hard mining for imbalanced deep learning. In Proceedings of the IEEE international conference on computer vision, pages 1851–1860, 2017.
  14. Deep miml network. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
  15. Transfg: A transformer architecture for fine-grained recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 852–860, 2022.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  17. Support vector machines. IEEE Intelligent Systems and their applications, 13(4):18–28, 1998.
  18. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017.
  19. Patch-based convolutional neural network for whole slide tissue image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2424–2433, 2016.
  20. Attention-based deep multiple instance learning. In International conference on machine learning, pages 2127–2136. PMLR, 2018.
  21. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  22. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In CVPR, pages 14318–14328, 2021.
  23. Dt-mil: Deformable transformer for multi-instance learning on histopathological image. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 206–216. Springer, 2021.
  24. Deep instance-level hard negative mining model for histopathology images. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part I 22, pages 514–522. Springer, 2019.
  25. Ai-based pathology predicts origins for cancers of unknown primary. Nature, 594(7861):106–110, 2021.
  26. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering, 5(6):555–570, 2021.
  27. Subcategory classifiers for multiple-instance learning and its application to retinal nerve fiber layer visibility classification. IEEE Transactions on Medical Imaging, 36(5):1140–1150, 2017.
  28. A framework for multiple-instance learning. Advances in neural information processing systems, 10, 1997.
  29. Streaming convolutional neural networks for end-to-end learning with multi-megapixel images. IEEE transactions on pattern analysis and machine intelligence, 44(3):1581–1590, 2020.
  30. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 815–823, 2015.
  31. Transmil: Transformer based correlated multiple instance learning for whole slide image classification. NeurIPS, 34, 2021.
  32. Cluster-to-conquer: A framework for end-to-end multi-instance learning for whole slide image classification. arXiv preprint arXiv:2103.10626, 2021.
  33. Mining hard samples globally and efficiently for person reidentification. IEEE Internet of Things Journal, 7(10):9611–9622, 2020.
  34. Training region-based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 761–769, 2016.
  35. Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. Advances in neural information processing systems, 29, 2016.
  36. Deep neural network models for computational histopathology: A survey. Medical Image Analysis, 67:101813, 2021.
  37. Stochastic class-based hard example mining for deep metric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7251–7259, 2019.
  38. Mvp matching: A maximum-value perfect matching for mining hard samples, with application to person re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6737–6747, 2019.
  39. Cross-batch hard example mining with pseudo large batch for id vs. spot face recognition. IEEE Transactions on Image Processing, 31:3224–3235, 2022.
  40. Neural image compression for gigapixel histopathology image analysis. IEEE transactions on pattern analysis and machine intelligence, 43(2):567–578, 2019.
  41. Multiple instance learning for classification of dementia in brain mri. Medical Image Analysis, 18(5):808–818, 2014.
  42. Towards human-machine cooperation: Self-supervised sample mining for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1605–1613, 2018.
  43. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE transactions on cybernetics, 50(9):3950–3962, 2019.
  44. Revisiting multiple instance neural networks. Pattern Recognition, 74:15–24, 2018.
  45. Combining attention-based multiple instance learning and gaussian processes for ct hemorrhage detection. In MICCAI, pages 582–591. Springer, 2021.
  46. Camel: A weakly supervised learning framework for histopathology image segmentation. In Proceedings of the IEEE/CVF International Conference on computer vision, pages 10682–10691, 2019.
  47. Learning with batch-wise optimal transport loss for 3d shape recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3333–3342, 2019.
  48. Weakly supervised histopathology cancer image segmentation and classification. Medical Image Analysis, 18(3):591–604, 2014.
  49. Dtfd-mil: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18802–18812, 2022.
  50. Dual space multiple instance representative learning for medical image classification. In 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21-24, 2022. BMVA Press, 2022.
  51. Setmil: spatial encoding transformer-based multiple instance learning for pathological image analysis. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part II, pages 66–76. Springer, 2022.
Citations (31)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now