BoNuS: Boundary Mining for Nuclei Segmentation with Partial Point Labels (2401.07437v1)
Abstract: Nuclei segmentation is a fundamental prerequisite in the digital pathology workflow. The development of automated methods for nuclei segmentation enables quantitative analysis of the wide existence and large variances in nuclei morphometry in histopathology images. However, manual annotation of tens of thousands of nuclei is tedious and time-consuming, which requires significant amount of human effort and domain-specific expertise. To alleviate this problem, in this paper, we propose a weakly-supervised nuclei segmentation method that only requires partial point labels of nuclei. Specifically, we propose a novel boundary mining framework for nuclei segmentation, named BoNuS, which simultaneously learns nuclei interior and boundary information from the point labels. To achieve this goal, we propose a novel boundary mining loss, which guides the model to learn the boundary information by exploring the pairwise pixel affinity in a multiple-instance learning manner. Then, we consider a more challenging problem, i.e., partial point label, where we propose a nuclei detection module with curriculum learning to detect the missing nuclei with prior morphological knowledge. The proposed method is validated on three public datasets, MoNuSeg, CPM, and CoNIC datasets. Experimental results demonstrate the superior performance of our method to the state-of-the-art weakly-supervised nuclei segmentation methods. Code: https://github.com/hust-linyi/bonus.
- J. G. Elmore, G. M. Longton, P. A. Carney, B. M. Geller, T. Onega, A. N. Tosteson, H. D. Nelson, M. S. Pepe, K. H. Allison, S. J. Schnitt et al., “Diagnostic concordance among pathologists interpreting breast biopsy specimens,” Jama, vol. 313, no. 11, pp. 1122–1132, 2015.
- S. Graham, Q. D. Vu, S. E. A. Raza, A. Azam, Y. W. Tsang, J. T. Kwak, and N. Rajpoot, “Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images,” Medical Image Analysis, vol. 58, p. 101563, 2019.
- H. Chang, J. Han, A. Borowsky, L. Loss, J. W. Gray, P. T. Spellman, and B. Parvin, “Invariant delineation of nuclear architecture in glioblastoma multiforme for clinical and molecular association,” IEEE Transactions on Medical Imaging, vol. 32, no. 4, pp. 670–682, 2012.
- N. Kumar, R. Verma, S. Sharma, S. Bhargava, A. Vahadane, and A. Sethi, “A dataset and a technique for generalized nuclear segmentation for computational pathology,” IEEE Transactions on Medical Imaging, vol. 36, no. 7, pp. 1550–1560, 2017.
- H. Chen, X. Qi, L. Yu, Q. Dou, J. Qin, and P.-A. Heng, “DCAN: Deep contour-aware networks for object instance segmentation from histology images,” Medical Image Analysis, vol. 36, pp. 135–146, 2017.
- Y. Zhou, O. F. Onder, Q. Dou, E. Tsougenis, H. Chen, and P.-A. Heng, “CIA-Net: Robust nuclei instance segmentation with contour-aware information aggregation,” in International Conference on Information Processing in Medical Imaging. Springer, 2019, pp. 682–693.
- A. R. Jara-Lazaro, T. P. Thamboo, M. Teh, and P. H. Tan, “Digital pathology: exploring its applications in diagnostic surgical pathology practice,” Pathology, vol. 42, no. 6, pp. 512–518, 2010.
- H. Qu, P. Wu, Q. Huang, J. Yi, Z. Yan, K. Li, G. M. Riedlinger, S. De, S. Zhang, and D. N. Metaxas, “Weakly supervised deep nuclei segmentation using partial points annotation in histopathology images,” IEEE Transactions on Medical Imaging, vol. 39, no. 11, pp. 3655–3666, 2020.
- M. Z. Alom, C. Yakopcic, T. M. Taha, and V. K. Asari, “Microscopic nuclei classification, segmentation and detection with improved deep convolutional neural network (DCNN) approaches,” arXiv preprint arXiv:1811.03447, 2018.
- K. Q. Brown, “Voronoi diagrams from convex hulls,” Information Processing Letters, vol. 9, no. 5, pp. 223–228, 1979.
- P. Naylor, M. Laé, F. Reyal, and T. Walter, “Segmentation of nuclei in histopathology images by deep regression of the distance map,” IEEE Transactions on Medical Imaging, vol. 38, no. 2, pp. 448–459, 2018.
- X. Xie, J. Chen, Y. Li, L. Shen, K. Ma, and Y. Zheng, “Instance-aware self-supervised learning for nuclei segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 2020, pp. 341–350.
- Y. Lin, Z. Qu, H. Chen, Z. Gao, Y. Li, L. Xia, K. Ma, Y. Zheng, and K.-T. Cheng, “Nuclei segmentation with point annotations from pathology images via self-supervised learning and co-training,” Medical Image Analysis, vol. 89, p. 102933, 2023.
- Q. D. Vu, S. Graham, T. Kurc, M. N. N. To, M. Shaban, T. Qaiser, N. A. Koohbanani, S. A. Khurram, J. Kalpathy-Cramer, T. Zhao et al., “Methods for segmentation and classification of digital microscopy tissue images,” Frontiers in Bioengineering and Biotechnology, p. 53, 2019.
- S. Graham, M. Jahanifar, Q. D. Vu, G. Hadjigeorghiou, T. Leech, D. Snead, S. E. A. Raza, F. Minhas, and N. Rajpoot, “Conic: Colon nuclei identification and counting challenge 2022,” arXiv preprint arXiv:2111.14485, 2021.
- X. Yang, H. Li, and X. Zhou, “Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and kalman filter in time-lapse microscopy,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 53, no. 11, pp. 2405–2414, 2006.
- S. Ali and A. Madabhushi, “An integrated region-, boundary-, shape-based active contour for multiple object overlap resolution in histological imagery,” IEEE Transactions on Medical Imaging, vol. 31, no. 7, pp. 1448–1460, 2012.
- M. Liao, Y.-q. Zhao, X.-h. Li, P.-s. Dai, X.-w. Xu, J.-k. Zhang, and B.-j. Zou, “Automatic segmentation for cell images based on bottleneck detection and ellipse fitting,” Neurocomputing, vol. 173, pp. 615–622, 2016.
- S. E. A. Raza, L. Cheung, M. Shaban, S. Graham, D. Epstein, S. Pelengaris, M. Khan, and N. M. Rajpoot, “Micro-Net: A unified model for segmentation of various objects in microscopy images,” Medical Image Analysis, vol. 52, pp. 160–173, 2019.
- O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 2015, pp. 234–241.
- N. A. Koohbanani, M. Jahanifar, N. Z. Tajadin, and N. Rajpoot, “Nuclick: a deep learning framework for interactive segmentation of microscopic images,” Medical Image Analysis, vol. 65, p. 101771, 2020.
- H. Chen, K. Sun, Z. Tian, C. Shen, Y. Huang, and Y. Yan, “Blendmask: Top-down meets bottom-up for instance segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 8573–8581.
- Y. Zhou, H. Chen, J. Xu, Q. Dou, and P.-A. Heng, “IRNet: Instance relation network for overlapping cervical cell segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 2019, pp. 640–648.
- X. Gong, S. Chen, B. Zhang, and D. Doermann, “Style consistent image generation for nuclei instance segmentation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 3994–4003.
- K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2961–2969.
- V. Kulharia, S. Chandra, A. Agrawal, P. Torr, and A. Tyagi, “Box2seg: Attention weighted loss and discriminative feature learning for weakly supervised segmentation,” in European Conference on Computer Vision. Springer, 2020, pp. 290–308.
- J. Pont-Tuset, P. Arbelaez, J. T. Barron, F. Marques, and J. Malik, “Multiscale combinatorial grouping for image segmentation and object proposal generation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 1, pp. 128–140, 2016.
- C. Rother, V. Kolmogorov, and A. Blake, “Interactive foreground extraction using iterated graph cuts,” ACM Transactions on Graphics, vol. 23, p. 3, 2012.
- M. Tang, A. Djelouah, F. Perazzi, Y. Boykov, and C. Schroers, “Normalized cut loss for weakly-supervised CNN segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1818–1827.
- R. Qian, Y. Wei, H. Shi, J. Li, J. Liu, and T. Huang, “Weakly supervised scene parsing with point-based distance metric learning,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 8843–8850.
- J. Ahn and S. Kwak, “Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation,” in IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4981–4990.
- D. Zhang, J. Han, G. Cheng, and M.-H. Yang, “Weakly supervised object localization and detection: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, pp. 5866–5885, 2021.
- Y. Ouassit, S. Ardchir, M. Yassine El Ghoumari, and M. Azouazi, “A brief survey on weakly supervised semantic segmentation.” International Journal of Online & Biomedical Engineering, vol. 18, no. 10, 2022.
- C.-C. Hsu, K.-J. Hsu, C.-C. Tsai, Y.-Y. Lin, and Y.-Y. Chuang, “Weakly supervised instance segmentation using the bounding box tightness prior,” Advances in Neural Information Processing Systems, vol. 32, 2019.
- J. Wang and B. Xia, “Bounding box tightness prior for weakly supervised image segmentation,” in International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 2021, pp. 526–536.
- H. Qu, P. Wu, Q. Huang, J. Yi, G. M. Riedlinger, S. De, and D. N. Metaxas, “Weakly supervised deep nuclei segmentation using points annotation in histopathology images,” in International Conference on Medical Imaging with Deep Learning. PMLR, 2019, pp. 390–400.
- A. Chamanzar and Y. Nie, “Weakly supervised multi-task learning for cell detection and segmentation,” in IEEE International Symposium on Biomedical Imaging. IEEE, 2020, pp. 513–516.
- I. Yoo, D. Yoo, and K. Paeng, “PseudoEdgeNet: Nuclei segmentation only with point annotations,” in International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 2019, pp. 731–739.
- K. Tian, J. Zhang, H. Shen, K. Yan, P. Dong, J. Yao, S. Che, P. Luo, and X. Han, “Weakly-supervised nucleus segmentation based on point annotations: A coarse-to-fine self-stimulated learning strategy,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2020, pp. 299–308.
- J. A. Hartigan and M. A. Wong, “Algorithm as 136: A k-means clustering algorithm,” Journal of the Royal Statistical Society. Series C (Applied Statistics), vol. 28, no. 1, pp. 100–108, 1979.
- T. D. Pereira, N. Tabris, A. Matsliah, D. M. Turner, J. Li, S. Ravindranath, E. S. Papadoyannis, E. Normand, D. S. Deutsch, Z. Y. Wang et al., “SLEAP: A deep learning system for multi-animal pose tracking,” Nature Methods, vol. 19, no. 4, pp. 486–495, 2022.
- Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in International Conference on Machine Learning, 2009, pp. 41–48.
- H. Zhang, Y. Meng, Y. Zhao, Y. Qiao, X. Yang, S. E. Coupland, and Y. Zheng, “DTFD-MIL: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18 802–18 812.
- L. Kostrykin and K. Rohr, “Superadditivity and convex optimization for globally optimal cell segmentation using deformable shape models,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.
- J. Ahn, S. Cho, and S. Kwak, “Weakly supervised learning of instance segmentation with inter-pixel relations,” in IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 2209–2218.
- X. Wang, S. Liu, H. Ma, and M.-H. Yang, “Weakly-supervised semantic segmentation by iterative affinity learning,” International Journal of Computer Vision, vol. 128, pp. 1736–1749, 2020.
- D. Zhang, Y. Lin, H. Chen, Z. Tian, X. Yang, J. Tang, and K. T. Cheng, “Deep learning for medical image segmentation: tricks, challenges and future directions,” arXiv preprint arXiv:2209.10307, 2022.
- Y. Lin, D. Zhang, X. Fang, Y. Chen, K.-T. Cheng, and H. Chen, “Rethinking boundary detection in deep learning models for medical image segmentation,” in International Conference on Information Processing in Medical Imaging. Springer, 2023, pp. 730–742.
- H. Lee and W.-K. Jeong, “Scribble2Label: Scribble-supervised cell segmentation via self-generating pseudo-labels with consistency,” in International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 2020, pp. 14–23.
- K. Sirinukunwattana, D. R. J. Snead, and N. M. Rajpoot, “A stochastic polygons model for glandular structures in colon histology images,” IEEE Transactions on Medical Imaging, vol. 34, no. 11, pp. 2366–2378, 2015.
- M. Zhang, Y. Zhou, J. Zhao, Y. Man, B. Liu, and R. Yao, “A survey of semi-and weakly supervised semantic segmentation of images,” Artificial Intelligence Review, vol. 53, pp. 4259–4288, 2020.