Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 69 tok/s
Gemini 2.5 Pro 58 tok/s Pro
GPT-5 Medium 32 tok/s Pro
GPT-5 High 29 tok/s Pro
GPT-4o 108 tok/s Pro
Kimi K2 198 tok/s Pro
GPT OSS 120B 461 tok/s Pro
Claude Sonnet 4.5 33 tok/s Pro
2000 character limit reached

MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Segmentation Based on 3D Masked Autoencoding and Pseudo-Labeling (2303.09373v3)

Published 16 Mar 2023 in cs.CV, cs.AI, and cs.LG

Abstract: Robust segmentation is critical for deriving quantitative measures from large-scale, multi-center, and longitudinal medical scans. Manually annotating medical scans, however, is expensive and labor-intensive and may not always be available in every domain. Unsupervised domain adaptation (UDA) is a well-studied technique that alleviates this label-scarcity problem by leveraging available labels from another domain. In this study, we introduce Masked Autoencoding and Pseudo-Labeling Segmentation (MAPSeg), a $\textbf{unified}$ UDA framework with great versatility and superior performance for heterogeneous and volumetric medical image segmentation. To the best of our knowledge, this is the first study that systematically reviews and develops a framework to tackle four different domain shifts in medical image segmentation. More importantly, MAPSeg is the first framework that can be applied to $\textbf{centralized}$, $\textbf{federated}$, and $\textbf{test-time}$ UDA while maintaining comparable performance. We compare MAPSeg with previous state-of-the-art methods on a private infant brain MRI dataset and a public cardiac CT-MRI dataset, and MAPSeg outperforms others by a large margin (10.5 Dice improvement on the private MRI dataset and 5.7 on the public CT-MRI dataset). MAPSeg poses great practical value and can be applied to real-world problems. GitHub: https://github.com/XuzheZ/MAPSeg/.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (81)
  1. A new neonatal cortical and subcortical brain atlas: the melbourne children’s regional infant brain (m-crib) atlas. Neuroimage, 147:841–851, 2017.
  2. Structural neuroimaging correlates of social deficits are similar in autism spectrum disorder and attention-deficit/hyperactivity disorder: analysis from the pond network. Translational psychiatry, 9(1):72, 2019.
  3. Synergistic image and feature adaptation: Towards cross-modality domain adaptation for medical image segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):865–872, 2019a.
  4. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE Transactions on Medical Imaging, 39(7):2494–2505, 2020a.
  5. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587, 2017a.
  6. Generative pretraining from pixels. In Proceedings of the 37th International Conference on Machine Learning, pages 1691–1703. PMLR, 2020b.
  7. Collaborative global-local networks for memory-efficient segmentation of ultra-high resolution images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019b.
  8. No more discrimination: Cross city adaptation of road scene segmenters. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 2011–2020, Los Alamitos, CA, USA, 2017b. IEEE Computer Society.
  9. Domain adaptation in the absence of source domain data. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, page 451–460, New York, NY, USA, 2016. Association for Computing Machinery.
  10. Cortical-subcortical interactions in goal-directed behavior. Physiological reviews, 103(1):347–389, 2023.
  11. Structure-driven unsupervised domain adaptation for cross-modality cardiac segmentation. IEEE Transactions on Medical Imaging, 40(12):3604–3616, 2021.
  12. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015.
  13. Partial unbalanced feature transport for cross-modality cardiac image segmentation. IEEE Transactions on Medical Imaging, 42(6):1758–1773, 2023.
  14. Pnp-adanet: Plug-and-play adversarial domain adaptation network at unpaired cross-modality cardiac segmentation. IEEE Access, 7:99065–99076, 2019.
  15. Ssf-dan: Separated semantic feature based domain adaptation network for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
  16. The developing human connectome project neonatal data release. Frontiers in neuroscience, 16, 2022.
  17. Unsupervised domain adaptation for medical image segmentation by selective entropy constraints and adaptive semantic alignment. Proceedings of the AAAI Conference on Artificial Intelligence, 37(1):623–631, 2023.
  18. Imaging structural and functional brain development in early childhood. Nature Reviews Neuroscience, 19(3):123–137, 2018.
  19. Semi-supervised learning by entropy minimization. In Advances in Neural Information Processing Systems. MIT Press, 2004.
  20. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271–21284, 2020.
  21. Federated learning for medical image analysis: A survey. CoRR, abs/2306.05980, 2023.
  22. Early brain development in infants at high risk for autism spectrum disorder. Nature, 542(7641):348–351, 2017.
  23. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  24. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009, 2022.
  25. Olivier Henaff. Data-efficient image recognition with contrastive predictive coding. In Proceedings of the 37th International Conference on Machine Learning, pages 4182–4192. PMLR, 2020.
  26. CyCADA: Cycle-consistent adversarial domain adaptation. In Proceedings of the 35th International Conference on Machine Learning, pages 1989–1998. PMLR, 2018.
  27. Synthstrip: skull-stripping for any brain image. NeuroImage, 260:119474, 2022.
  28. The unc/umn baby connectome project (bcp): An overview of the study design and protocol development. NeuroImage, 185:891–905, 2019.
  29. Daformer: Improving network architectures and training strategies for domain-adaptive semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9924–9935, 2022a.
  30. Hrda: Context-aware high-resolution domain-adaptive semantic segmentation. In Computer Vision – ECCV 2022, pages 372–391, Cham, 2022b. Springer Nature Switzerland.
  31. Mic: Masked image consistency for context-enhanced domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11721–11732, 2023.
  32. Simple: Similar pseudo label exploitation for semi-supervised classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15099–15108, 2021.
  33. Contextual-relation consistent domain adaptation for semantic segmentation. In Computer Vision – ECCV 2020, pages 705–722, Cham, 2020. Springer International Publishing.
  34. Wen Ji and Albert C. S. Chung. Unsupervised domain adaptation for medical image segmentation using transformer with meta attention. IEEE Transactions on Medical Imaging, pages 1–1, 2023.
  35. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021.
  36. What to hide from your students: Attention-guided masked image modeling. In Computer Vision – ECCV 2022, pages 300–318, Cham, 2022. Springer Nature Switzerland.
  37. Understanding masked image modeling via learning occlusion invariant feature. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6241–6251, 2023.
  38. Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242, 2016.
  39. Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, page 896. Atlanta, 2013.
  40. Fedmask: Joint computation and communication-efficient personalized federated learning via heterogeneous masking. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, page 42–55, New York, NY, USA, 2021. Association for Computing Machinery.
  41. Reducing domain gap in frequency and spatial domain for cross-modality domain adaptation on medical image segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 37(2):1719–1727, 2023a.
  42. A structure-aware framework of unsupervised cross-modality domain adaptation via frequency and spatial knowledge distillation. IEEE Transactions on Medical Imaging, pages 1–1, 2023b.
  43. Source-free domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1215–1224, 2021.
  44. Margin preserving self-paced contrastive learning towards domain adaptation for medical image segmentation. IEEE Journal of Biomedical and Health Informatics, 26(2):638–647, 2022.
  45. Learning transferable features with deep adaptation networks. In Proceedings of the 32nd International Conference on Machine Learning, pages 97–105, Lille, France, 2015. PMLR.
  46. Unsupervised domain adaptation with residual transfer networks. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2016.
  47. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations, 2017.
  48. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.
  49. 𝒳𝒳\mathcal{X}caligraphic_X-metric: An n-dimensional information-theoretic framework for groupwise registration and deep combined computing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(7):9206–9224, 2023.
  50. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017.
  51. Federated alternate training (fat): Leveraging unannotated data silos in federated segmentation for medical imaging. In 20th IEEE International Symposium on Biomedical Imaging, ISBI 2023, Cartagena, Colombia, April 18-21, 2023, pages 1–5. IEEE, 2023.
  52. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  53. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pages 1406–1415, 2019.
  54. Exploiting pseudo labels in a self-supervised learning framework for improved monocular depth estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1578–1588, 2022.
  55. Torchio: A python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Computer Methods and Programs in Biomedicine, 208:106236, 2021.
  56. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2016.
  57. Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, pages 92–104, Cham, 2019. Springer International Publishing.
  58. Subcortical brain development in autism and fragile x syndrome: evidence for dynamic, age-and disorder-specific trajectories in infancy. American Journal of Psychiatry, 179(8):562–572, 2022.
  59. Longitudinal segmentation of age-related white matter hyperintensities. Medical Image Analysis, 38:50–64, 2017.
  60. Multi-site infant brain segmentation algorithms: The iseg-2019 challenge. IEEE Transactions on Medical Imaging, 40(5):1363–1376, 2021.
  61. Fedcor: Correlation-based active client selection strategy for heterogeneous federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10102–10111, 2022a.
  62. Self-supervised pre-training of swin transformers for 3d medical image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20730–20740, 2022b.
  63. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2017.
  64. Privacy preservation in federated learning: Insights from the GDPR perspective. CoRR, abs/2011.05411, 2020.
  65. N4itk: improved n3 bias correction. IEEE transactions on medical imaging, 29(6):1310–1320, 2010.
  66. Automated model-based tissue classification of mr images of the brain. IEEE Transactions on Medical Imaging, 18(10):897–908, 1999.
  67. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2512–2521, Los Alamitos, CA, USA, 2019. IEEE Computer Society.
  68. Fedmix: Mixed supervised federated learning for medical image segmentation. IEEE Transactions on Medical Imaging, 42(7):1955–1968, 2023.
  69. Accurate and versatile 3d segmentation of plant tissues at cellular resolution. Elife, 9:e57613, 2020.
  70. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9653–9663, 2022.
  71. Revealing the dark secrets of masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14475–14485, 2023a.
  72. On data scaling in masked image modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10365–10374, 2023b.
  73. Federated semi-supervised learning for covid region segmentation in chest ct using multi-national data from china, italy, japan. Medical Image Analysis, 70:101992, 2021.
  74. Federated multi-target domain adaptation. In 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 1081–1090, 2022a.
  75. A novel 3d unsupervised domain adaptation framework for cross-modality medical image segmentation. IEEE Journal of Biomedical and Health Informatics, 26(10):4976–4986, 2022b.
  76. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
  77. Prototypical pseudo label denoising and target structure learning for domain adaptive semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12414–12424, 2021.
  78. Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. International Journal of Computer Vision, 129(4):1106–1120, 2021.
  79. Xiahai Zhuang. Multivariate mixture model for myocardial segmentation combining multi-source images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(12):2933–2946, 2019.
  80. Multi-scale patch and multi-modality atlases for whole heart segmentation of mri. Medical Image Analysis, 31:77–87, 2016.
  81. Confidence regularized self-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 2 posts and received 2 likes.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up to View