Papers
Topics
Authors
Recent
Search
2000 character limit reached

Inter- and intra-uncertainty based feature aggregation model for semi-supervised histopathology image segmentation

Published 19 Mar 2024 in cs.CV | (2403.12767v1)

Abstract: Acquiring pixel-level annotations is often limited in applications such as histology studies that require domain expertise. Various semi-supervised learning approaches have been developed to work with limited ground truth annotations, such as the popular teacher-student models. However, hierarchical prediction uncertainty within the student model (intra-uncertainty) and image prediction uncertainty (inter-uncertainty) have not been fully utilized by existing methods. To address these issues, we first propose a novel inter- and intra-uncertainty regularization method to measure and constrain both inter- and intra-inconsistencies in the teacher-student architecture. We also propose a new two-stage network with pseudo-mask guided feature aggregation (PG-FANet) as the segmentation model. The two-stage structure complements with the uncertainty regularization strategy to avoid introducing extra modules in solving uncertainties and the aggregation mechanisms enable multi-scale and multi-stage feature integration. Comprehensive experimental results over the MoNuSeg and CRAG datasets show that our PG-FANet outperforms other state-of-the-art methods and our semi-supervised learning framework yields competitive performance with a limited amount of labeled data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (86)
  1. Nuclei segmentation with recurrent residual convolutional neural networks based U-Net (R2U-Net). In NAECON 2018-IEEE National Aerospace and Electronics Conference (pp. 228–233). IEEE.
  2. Glandular morphometrics for objective grading of colorectal adenocarcinoma histology images. Scientific Reports, 7, 1–12.
  3. Bidirectional Copy-Paste for Semi-Supervised Medical Image Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11514–11524).
  4. Pseudo-label guided contrastive learning for semi-supervised medical image segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 19786–19797).
  5. Auto-DenseUNet: Searchable neural network architecture for mass segmentation in 3D automated breast ultrasound. Medical Image Analysis, 82, 102589.
  6. Local contrastive loss with pseudo-label based self-training for semi-supervised medical image segmentation. Medical Image Analysis, 87, 102792.
  7. LinkNet: Exploiting encoder representations for efficient semantic segmentation. In 2017 IEEE Visual Communications and Image Processing (VCIP) (pp. 1–4). IEEE.
  8. DCAN: deep contour-aware networks for accurate gland segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2487–2496).
  9. Enhancing Nucleus Segmentation with HARU-Net: A Hybrid Attention Based Residual U-Blocks Network. arXiv preprint arXiv:2308.03382, .
  10. Searching for efficient multi-scale architectures for dense image prediction. In Advances in Neural Information Processing Systems (pp. 8699–8710).
  11. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 834–848.
  12. Semi-supervised semantic segmentation with cross pseudo supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2613–2622).
  13. DeepMC-iNABP: Deep learning for multiclass identification and classification of nucleic acid-binding proteins. Computational and Structural Biotechnology Journal, 20, 2020–2028.
  14. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition (pp. 248–255). IEEE.
  15. PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network at Unpaired Cross-Modality Cardiac Segmentation. IEEE Access, 7, 99065–99076. doi:10.1109/ACCESS.2019.2929258.
  16. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning (pp. 1050–1059).
  17. A data-scalable transformer for medical image segmentation: architecture, model efficiency, and benchmark. arXiv preprint arXiv:2203.00131, .
  18. MILD-Net: minimal information loss dilated network for gland instance segmentation in colon histology images. Medical Image Analysis, 52, 199–211.
  19. Contrastive semi-supervised learning for domain adaptive segmentation across similar anatomical structures. IEEE Transactions on Medical Imaging, 42, 245–256.
  20. Evaluating scalable Bayesian deep learning methods for robust computer vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 318–319).
  21. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1026–1034).
  22. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770–778).
  23. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Networks, 121, 74–87.
  24. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, .
  25. Cellular community detection for tissue phenotyping in colorectal cancer histology images. Medical Image Analysis, 63, 101696.
  26. UXNet: Searching Multi-level Feature Aggregation for 3D Medical Image Segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 346–356). Springer.
  27. DoNet: Deep De-overlapping Network for Cytology Instance Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 15641–15650).
  28. Semi-supervised histological image segmentation via hierarchical consistency enforcement. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 3–13). Springer.
  29. What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (pp. 5574–5584).
  30. A multi-organ nucleus segmentation challenge. IEEE Transactions on Medical Imaging, 39, 1380–1391.
  31. Uncertainty quantification using Bayesian neural networks in classification: Application to biomedical image segmentation. Computational Statistics & Data Analysis, 142, 106816.
  32. Semi-supervised medical image segmentation using adversarial consistency learning and dynamic convolution network. IEEE Transactions on Medical Imaging, .
  33. DFANet: Deep feature aggregation for real-time semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 9522–9531).
  34. Dual-teacher: Integrating intra-domain and inter-domain teachers for annotation-efficient cardiac segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 418–427). Springer.
  35. Transformation-consistent self-ensembling model for semi-supervised medical image segmentation. IEEE Transactions on Neural Networks and Learning Systems, (pp. 1–12).
  36. Self-loop uncertainty: A novel pseudo-label for semi-supervised medical image segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 614–623). Springer.
  37. NPCNet: jointly segment primary nasopharyngeal carcinoma tumors and metastatic lymph nodes in MR images. IEEE Transactions on Medical Imaging, 41, 1639–1650.
  38. Nuclei Segmentation via a Deep Panoptic Model with Semantic Feature Fusion. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19 (pp. 861–868). doi:10.24963/ijcai.2019/121.
  39. Feature-driven local cell graph (FLocK): new computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers. Medical Image Analysis, 68, 101903.
  40. Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency. Medical Image Analysis, 80, 102517.
  41. M-Net: A convolutional neural network for deep brain structure segmentation. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) (pp. 437–440). IEEE.
  42. Bayesian Networks and Decision Graphs. Springer Science & Business Media.
  43. U2-Net: Going deeper with nested U-structure for salient object detection. Pattern Recognition, 106, 107404.
  44. Improving nuclei/gland instance segmentation in histopathology images by full resolution neural network and spatial constrained loss. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 378–386). Springer.
  45. Micro-Net: A unified model for segmentation of various objects in microscopy images. Medical Image Analysis, 52, 160–173.
  46. U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 234–241). Springer.
  47. Semi-supervised pixel contrastive learning framework for tissue segmentation in histopathological image. IEEE Journal of Biomedical and Health Informatics, 27, 97–108.
  48. Robust cell detection and segmentation in histopathological images using sparse reconstruction and stacked denoising autoencoders. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 383–390). Springer.
  49. Triplanar ensemble U-Net model for white matter hyperintensities segmentation on MR images. Medical Image Analysis, 73, 102184.
  50. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems (pp. 1195–1204).
  51. Interpolation Consistency Training for Semi-supervised Learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence IJCAI’19 (pp. 3635–3641). AAAI Press.
  52. Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing, 338, 34–45.
  53. Tripled-uncertainty guided mean teacher model for semi-supervised medical image segmentation. In Medical Image Computing and Computer Assisted Intervention (pp. 450–460). Springer.
  54. Semi-supervised medical image segmentation via a tripled-uncertainty guided mean teacher model with contrastive learning. Medical Image Analysis, 79, 102447.
  55. Dual Multiscale Mean Teacher Network for Semi-Supervised Infection Segmentation in Chest CT Volume for COVID-19. IEEE Transactions on Cybernetics, 53, 6363–6375.
  56. Double-Uncertainty Weighted Method for Semi-supervised Learning. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 542–551). Springer.
  57. 3D segmentation with exponential logarithmic loss for highly unbalanced object sizes. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 612–619). Springer.
  58. Cross-patch dense contrastive learning for semi-supervised segmentation of cellular nuclei in histopathologic images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11666–11675).
  59. Mutual consistency learning for semi-supervised medical image segmentation. Medical Image Analysis, 81, 102530.
  60. BiO-Net: Learning Recurrent Bi-directional Connections for Encoder-Decoder Architecture. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 74–84). Springer.
  61. Towards bi-directional skip connections in encoder-decoder architectures and beyond. Medical Image Analysis, 78, 102420.
  62. Deep Segmentation-Emendation Model for Gland Instance Segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 469–477). Springer.
  63. Pairwise Relation Learning for Semi-supervised Gland Segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 417–427). Springer.
  64. BMAnet: Boundary mining with adversarial learning for semi-supervised 2D myocardial infarction segmentation. IEEE Journal of Biomedical and Health Informatics, 27, 87–96.
  65. Asymmetric multi-task attention network for prostate bed segmentation in computed tomography images. Medical Image Analysis, 72, 102116.
  66. Ambiguity-selective consistency regularization for mean-teacher semi-supervised medical image segmentation. Medical Image Analysis, 88, 102880.
  67. PreTP-2L: Identification of therapeutic peptides and their types using two-layer ensemble learning framework. Bioinformatics, 39, btad125.
  68. sAMPpred-GAT: prediction of antimicrobial peptide by graph attention network and predicted peptide structure. Bioinformatics, 39, btac715.
  69. ADS_UNet: A nested UNet for histopathology image segmentation. Expert Systems with Applications, 226, 120128.
  70. Directional connectivity-based segmentation of medical images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11525–11535).
  71. Context prior for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12416–12425).
  72. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, .
  73. Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 605–613). Springer.
  74. UNesT: local spatial representation learning with hierarchical transformer for efficient medical segmentation. Medical Image Analysis, 90, 102939.
  75. Attention Residual Learning for Skin Lesion Classification. IEEE Transactions on Medical Imaging, 38, 2092–2103.
  76. Multi-modal contrastive mutual learning and pseudo-label re-learning for semi-supervised medical image segmentation. Medical Image Analysis, 83, 102656.
  77. Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 408–416). Springer.
  78. Triple U-Net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation. Medical Image Analysis, 65, 101786.
  79. RCPS: Rectified contrastive pseudo supervision for semi-supervised medical image segmentation. arXiv preprint arXiv:2301.05500, .
  80. Cartilage segmentation in high-resolution 3D micro-CT images via uncertainty-guided self-training with very sparse annotation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 802–812). Springer.
  81. A new ensemble learning framework for 3D biomedical image segmentation. In Proceedings of the AAAI Conference on Artificial Intelligence (pp. 5909–5916). volume 33.
  82. Double noise mean teacher self-ensembling model for semi-supervised tumor segmentation. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1446–1450). IEEE.
  83. Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. International Journal of Computer Vision, 129, 1106–1120.
  84. Squeeze-and-attention networks for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13065–13074).
  85. Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation. In International Conference on Medical Image Computing and Computer Assisted Intervention (pp. 521–531). Springer.
  86. Hybrid Dual Mean-Teacher Network With Double-Uncertainty Guidance for Semi-Supervised Segmentation of MRI Scans. arXiv preprint arXiv:2303.05126, .
Citations (11)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up