Papers
Topics
Authors
Recent
Search
2000 character limit reached

Harmonizing Flows: Leveraging normalizing flows for unsupervised and source-free MRI harmonization

Published 22 Jul 2024 in cs.CV | (2407.15717v1)

Abstract: Lack of standardization and various intrinsic parameters for magnetic resonance (MR) image acquisition results in heterogeneous images across different sites and devices, which adversely affects the generalization of deep neural networks. To alleviate this issue, this work proposes a novel unsupervised harmonization framework that leverages normalizing flows to align MR images, thereby emulating the distribution of a source domain. The proposed strategy comprises three key steps. Initially, a normalizing flow network is trained to capture the distribution characteristics of the source domain. Then, we train a shallow harmonizer network to reconstruct images from the source domain via their augmented counterparts. Finally, during inference, the harmonizer network is updated to ensure that the output images conform to the learned source domain distribution, as modeled by the normalizing flow network. Our approach, which is unsupervised, source-free, and task-agnostic is assessed in the context of both adults and neonatal cross-domain brain MRI segmentation, as well as neonatal brain age estimation, demonstrating its generalizability across tasks and population demographics. The results underscore its superior performance compared to existing methodologies. The code is available at https://github.com/farzad-bz/Harmonizing-Flows

Definition Search Book Streamline Icon: https://streamlinehq.com
References (75)
  1. Hidemasa Takao et al. Effect of scanner in longitudinal studies of brain volume changes. Journal of Magnetic Resonance Imaging, 34(2):438–444, 2011.
  2. Blake E Dewey et al. Deepharmony: A deep learning approach to contrast harmonization across scanner changes. Magnetic resonance imaging, 64:160–170, 2019.
  3. Diffusion models for contrast harmonization of magnetic resonance images. arXiv preprint arXiv:2303.08189, 2023.
  4. Raymond Pomponio et al. Harmonization of large MRI datasets for the analysis of brain imaging patterns throughout the lifespan. NeuroImage, 208:116450, 2020.
  5. Gourav Modanwal et al. MRI image harmonization using cycle-consistent generative adversarial network. In SPIE Medical Imaging 2020, volume 11314, pages 259–264, 2020.
  6. Mengting Liu et al. Style transfer using generative adversarial networks for multi-site MRI harmonization. In MICCAI, pages 313–322, 2021.
  7. Imunity: a generalizable vae-gan solution for multicenter mr image harmonization. Medical Image Analysis, 88:102799, 2023.
  8. Pierre-Luc Delisle et al. Realistic image normalization for multi-Domain segmentation. Medical Image Analysis, 74:102191, 2021.
  9. Nicola K Dinsdale et al. Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal. NeuroImage, 228:117689, 2021.
  10. Neerav Karani et al. Test-time adaptable neural networks for robust medical image segmentation. Medical Image Analysis, 68:101907, 2021.
  11. Harmonizing flows: Unsupervised MR harmonization based on normalizing flows. In International Conference on Information Processing in Medical Imaging, pages 347–359. Springer, 2023.
  12. New variants of a method of MRI scale standardization. IEEE transactions on medical imaging, 19(2):143–150, 2000.
  13. Russell Shinohara et al. Statistical normalization techniques for magnetic resonance imaging. NeuroImage: Clinical, 6:9–19, 2014.
  14. Jean-Philippe Fortin et al. Removing inter-subject technical variability in magnetic resonance imaging studies. NeuroImage, 132:198–212, 2016.
  15. Jean-Philippe Fortin et al. Harmonization of multi-site diffusion tensor imaging data. Neuroimage, 161:149–170, 2017.
  16. Joanne C Beer et al. Longitudinal Combat: A method for harmonizing longitudinal multi-scanner imaging data. Neuroimage, 220:117129, 2020.
  17. Jun-Yan Zhu et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, pages 2223–2232, 2017.
  18. Lianrui Zuo et al. Information-based disentangled representation learning for unsupervised MR harmonization. In IPMI, pages 346–359, 2021.
  19. Multi-scanner harmonization of paired neuroimaging data via structure preserving embedding learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3284–3293, 2021.
  20. Blake E Dewey et al. A disentangled latent space for cross-site MRI harmonization. In MICCAI, pages 720–729, 2020.
  21. Haca3: A unified approach for multi-site mr image harmonization. Computerized Medical Imaging and Graphics, 109:102285, 2023.
  22. Structural mri harmonization via disentangled latent energy-based style translation. In International Workshop on Machine Learning in Medical Imaging, pages 1–11. Springer, 2023.
  23. Shape-aware joint distribution alignment for cross-domain image segmentation. IEEE Transactions on Medical Imaging, 42(8):2338–2347, 2023.
  24. Cf distance: A new domain discrepancy metric and application to explicit domain adaptation for cross-modality cardiac image segmentation. IEEE Transactions on Medical Imaging, 39(12):4274–4285, 2020.
  25. Pnp-adanet: Plug-and-play adversarial domain adaptation network at unpaired cross-modality cardiac segmentation. IEEE Access, 7:99065–99076, 2019.
  26. Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. In Information Processing in Medical Imaging: 25th International Conference, IPMI 2017, Boone, NC, USA, June 25-30, 2017, Proceedings 25, pages 597–609. Springer, 2017.
  27. Unsupervised bidirectional cross-modality adaptation via deeply synergistic image and feature alignment for medical image segmentation. IEEE transactions on medical imaging, 39(7):2494–2505, 2020.
  28. MT-UDA: Towards unsupervised cross-modality medical image segmentation with limited source labels. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 293–303. Springer, 2021.
  29. Source-free domain adaptation for image segmentation. Medical Image Analysis, 82:102617, 2022.
  30. Source free domain adaptation for medical image segmentation with fourier style mining. Medical Image Analysis, 79:102457, 2022.
  31. Unsupervised model adaptation for source-free segmentation of medical images. Medical Image Analysis, 95:103179, 2024.
  32. Malik Boudiaf et al. Parameter-free Online Test-time Adaptation. In CVPR, pages 8344–8353, 2022.
  33. Chaithanya Kumar Mummadi et al. Test-time adaptation to distribution shift by confidence maximization and input transformation. ICLR, 2022.
  34. Jian Liang et al. Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation. In ICML, pages 6028–6039, 2020.
  35. Dequan Wang et al. TENT: Fully Test-Time Adaptation by Entropy Minimization. In ICLR, 2021.
  36. Efficient test-time model adaptation without forgetting. In International conference on machine learning, pages 16888–16905. PMLR, 2022.
  37. Towards stable test-time adaptation in dynamic wild world. arXiv preprint arXiv:2302.12400, 2023.
  38. Fully test-time adaptation for image segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part III 24, pages 251–260. Springer, 2021.
  39. Ian Goodfellow et al. Generative adversarial networks. Communications of the ACM, 63(11):139–144, 2020.
  40. Auto-Encoding Variational Bayes. stat, 1050:1, 2014.
  41. Improved techniques for training gans. NeurIPS, 29, 2016.
  42. Understanding posterior collapse in generative latent variable models. In DGS@ICLR, 2019.
  43. Laurent Dinh et al. Density estimation using real NVP. In ICLR, 2017.
  44. Jonathan Ho et al. Flow++: Improving flow-based generative models with variational dequantization and architecture design. In ICML, pages 2722–2730, 2019.
  45. Glow: Generative flow with invertible 1x1 convolutions. NeurIPS, 31, 2018.
  46. Noise flow: Noise modeling with conditional normalizing flows. In ICCV, pages 3165–3173, 2019.
  47. Moflow: an invertible flow model for generating molecular graphs. In Proceedings of the 26th ACM SIGKDD, pages 617–626, 2020.
  48. Denis Gudovskiy et al. Cflow-ad: Real-time unsupervised anomaly detection with localization via conditional normalizing flows. In WACV, pages 98–107, 2022.
  49. Why normalizing flows fail to detect out-of-distribution data. NeurIPS, 33:20578–20589, 2020.
  50. Aditya Grover et al. Alignflow: Cycle consistent learning from multiple domains via normalizing flows. In AAAI, pages 4028–4035, 2020.
  51. Ben Usman et al. Log-likelihood ratio minimizing flows: Towards robust and quantifiable neural distribution alignment. NeurIPS, 33:21118–21129, 2020.
  52. Tttflow: Unsupervised test-time training with normalizing flow. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2126–2134, 2023.
  53. Rongguang Wang et al. Harmonization with flow-based causal inference. In MICCAI, pages 181–190, 2021.
  54. Blindharmony:" blind" harmonization for mr images via flow model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21129–21139, 2023.
  55. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  56. Adriana Di Martino et al. The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Molecular psychiatry, 19(6):659–667, 2014.
  57. An open resource for transdiagnostic research in pediatric mental health and learning disorders. Scientific data, 4(1):1–26, 2017.
  58. Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of cognitive neuroscience, 19(9):1498–1507, 2007.
  59. 3D fully convolutional networks for subcortical segmentation in MRI: A large-scale study. NeuroImage, 170:456–470, 2018.
  60. Bruce Fischl. Freesurfer. Neuroimage, 62(2):774–781, 2012.
  61. The developing human connectome project: A minimal processing pipeline for neonatal cortical surface reconstruction. Neuroimage, 173:88–112, 2018.
  62. Neurobehaviour between birth and 40 weeks’ gestation in infants born< 30 weeks’ gestation and parental psychological wellbeing: predictors of brain development and child outcomes. BMC pediatrics, 14:1–13, 2014.
  63. Impact of moderate and late preterm birth on neurodevelopment, brain development and respiratory health at school age: protocol for a longitudinal cohort study (laprem study). BMJ open, 11(1):e044491, 2021.
  64. Parcellation of the neonatal cortex using surface-based melbourne children’s regional infant brain atlases (m-crib-s). Scientific reports, 10(1):4359, 2020.
  65. Fast image-level mri harmonization via spectrum analysis. In International Workshop on Machine Learning in Medical Imaging, pages 201–209. Springer, 2022.
  66. Guotai Wang et al. Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing, 338:34–45, 2019.
  67. Ling Zhang et al. Generalizing deep learning for medical image segmentation to unseen domains via deep stacked transformation. IEEE TMI, pages 2531–2540, 2020.
  68. Quantitative metrics for benchmarking medical image harmonization. In IEEE International Symposium on Biomedical Imaging (ISBI), 2024.
  69. Leonid V Kantorovich. Mathematical methods of organizing and planning production. Management science, 6(4):366–422, 1960.
  70. Fabian Isensee et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203–211, 2021.
  71. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  72. Milton Friedman. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the american statistical association, 32(200):675–701, 1937.
  73. Milton Friedman. A comparison of alternative tests of significance for the problem of m rankings. The annals of mathematical statistics, 11(1):86–92, 1940.
  74. Usb: A unified semi-supervised learning benchmark for classification. Advances in Neural Information Processing Systems, 35:3938–3961, 2022.
  75. Neighbor-aware calibration of segmentation networks with penalty-based constraints. arXiv preprint arXiv:2401.14487, 2024.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up