Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Contextual Embedding Learning to Enhance 2D Networks for Volumetric Image Segmentation (2404.01723v2)

Published 2 Apr 2024 in eess.IV and cs.CV

Abstract: The segmentation of organs in volumetric medical images plays an important role in computer-aided diagnosis and treatment/surgery planning. Conventional 2D convolutional neural networks (CNNs) can hardly exploit the spatial correlation of volumetric data. Current 3D CNNs have the advantage to extract more powerful volumetric representations but they usually suffer from occupying excessive memory and computation nevertheless. In this study we aim to enhance the 2D networks with contextual information for better volumetric image segmentation. Accordingly, we propose a contextual embedding learning approach to facilitate 2D CNNs capturing spatial information properly. Our approach leverages the learned embedding and the slice-wisely neighboring matching as a soft cue to guide the network. In such a way, the contextual information can be transferred slice-by-slice thus boosting the volumetric representation of the network. Experiments on challenging prostate MRI dataset (PROMISE12) and abdominal CT dataset (CHAOS) show that our contextual embedding learning can effectively leverage the inter-slice context and improve segmentation performance. The proposed approach is a plug-and-play, and memory-efficient solution to enhance the 2D networks for volumetric segmentation. Our code is publicly available at https://github.com/JuliusWang-7/CE_Block.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (47)
  1. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation. arXiv:1802.06955 .
  2. VoxResNet: Deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage 170, 446–455.
  3. DCAN: Deep contour-aware networks for accurate gland segmentation, in: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2487–2496.
  4. Combining fully convolutional and recurrent neural networks for 3D biomedical image segmentation. Advances in Neural Information Processing Systems 29.
  5. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587 .
  6. Blazingly fast video object segmentation with pixel-wise metric learning, in: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1189–1198.
  7. 3D U-Net: learning dense volumetric segmentation from sparse annotation, in: International Conference on Medical Image Computing and Computer-Assisted Interventions, pp. 424–432.
  8. FlowNet: Learning optical flow with convolutional networks, in: Proceedings of the IEEE Conference on Computer Vision (ICCV), pp. 2758–2766.
  9. Semantic instance segmentation via deep metric learning. arXiv preprint arXiv:1703.10277 .
  10. Automatic segmentation of prostate MRI using convolutional neural networks: Investigating the impact of network architecture on the accuracy of volume measurement and MRI-ultrasound registration. Medical Image Analysis 58, 101558.
  11. Intensity augmentation to improve generalizability of breast segmentation across different MRi scan protocols. IEEE Transactions on Biomedical Engineering 68, 759–770.
  12. Long short-term memory. Neural Computation 9, 1735–1780.
  13. Squeeze-and-excitation networks, in: IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141.
  14. Joint-phase attention network for breast cancer segmentation in DCE-MRI. Expert Systems with Applications 224, 119962.
  15. MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Networks 121, 74–87.
  16. ResUNet++: An advanced architecture for medical image segmentation, in: 2019 IEEE International Symposium on Multimedia (ISM), pp. 225–230.
  17. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis 36, 61–78.
  18. Deep learning. Nature 521, 436.
  19. H-DenseUNet: Hybrid densely connected unet for liver and tumor segmentation from CT volumes. IEEE Transactions on Medical Imaging 37, 2663–2674.
  20. Variance-aware attention U-net for multi-organ segmentation. Medical Physics 48, 7864–7876.
  21. Network in network. arXiv preprint arXiv:1312.4400 .
  22. A survey on deep learning in medical image analysis. Medical Image Analysis 42, 60–88.
  23. Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge. Medical Image Analysis 18, 359–373.
  24. V-net: Fully convolutional neural networks for volumetric medical image segmentation, in: 2016 Fourth International Conference on 3D Vision (3DV), IEEE. pp. 565–571.
  25. Deep sequential segmentation of organs in volumetric medical scans. IEEE Transactions on Medical Imaging 38, 1207–1215.
  26. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 .
  27. Current methods in medical image segmentation. Annual Review of Biomedical Engineering 2, 315–337.
  28. Recurrent fully convolutional neural networks for multi-slice MRI cardiac segmentation, in: Reconstruction, Segmentation, and Analysis of Medical Images. Springer, pp. 83–94.
  29. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network, in: Medical Image Computing and Computer-Assisted Interventions, Springer. pp. 246–253.
  30. U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical Image Computing and Computer-Assisted Interventions, Springer. pp. 234–241.
  31. Deep learning in medical image analysis. Annual Review of Biomedical Engineering 19, 221–248.
  32. Convolutional LSTM network: A machine learning approach for precipitation nowcasting, in: Advances in Neural Information Processing Systems, pp. 802–810.
  33. Inception-v4, inception-resnet and the impact of residual connections on learning, in: Thirty-first AAAI conference on artificial intelligence.
  34. N4ITK: improved N3 bias correction. IEEE Transactions on Medical Imaging 29, 1310–1320.
  35. Feelvos: Fast end-to-end embedding learning for video object segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9481–9490.
  36. A multi-view deep convolutional neural networks for lung nodule segmentation, in: International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE. pp. 1752–1755.
  37. Deep attentional features for prostate segmentation in ultrasound, in: Medical Image Computing and Computer Assisted Intervention, Springer. pp. 523–530.
  38. Deep attentive features for prostate segmentation in 3D transrectal ultrasound. IEEE Transactions on Medical Imaging 38, 2768–2778.
  39. 3D inception u-net with asymmetric loss for cancer detection in automated breast ultrasound. Medical Physics 47, 5582–5591.
  40. Deeply-supervised networks with threshold loss for cancer detection in automated breast ultrasound. IEEE Transactions on Medical Imaging 39, 866–876.
  41. Online robust projective dictionary learning: shape modeling for MR-TRUS registration. IEEE Transactions on Medical Imaging 37, 1067–1078.
  42. Learning pre-and post-contrast representation for breast cancer segmentation in DCE-MRI, in: International Symposium on Computer-Based Medical Systems, IEEE. pp. 355–359.
  43. Non-iterative scribble-supervised learning with pacing pseudo-masks for medical image segmentation. Expert Systems with Applications 238, 122024.
  44. Recurrent feature propagation and edge skip-connections for automatic abdominal organ segmentation. Expert Systems with Applications , 123856.
  45. Reciprocal learning for semi-supervised segmentation, in: Medical Image Computing and Computer Assisted Intervention, Springer. pp. 352–361.
  46. SimPLe: Similarity-aware propagation learning for weakly-supervised breast cancer segmentation in DCE-MRI, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 567–577.
  47. Unet++: A nested u-net architecture for medical image segmentation, in: Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Springer, pp. 3–11.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Zhuoyuan Wang (16 papers)
  2. Dong Sun (48 papers)
  3. Xiangyun Zeng (20 papers)
  4. Ruodai Wu (1 paper)
  5. Yi Wang (1038 papers)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets