Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration (2402.18933v2)

Published 29 Feb 2024 in cs.CV

Abstract: Establishing dense anatomical correspondence across distinct imaging modalities is a foundational yet challenging procedure for numerous medical image analysis studies and image-guided radiotherapy. Existing multi-modality image registration algorithms rely on statistical-based similarity measures or local structural image representations. However, the former is sensitive to locally varying noise, while the latter is not discriminative enough to cope with complex anatomical structures in multimodal scans, causing ambiguity in determining the anatomical correspondence across scans with different modalities. In this paper, we propose a modality-agnostic structural representation learning method, which leverages Deep Neighbourhood Self-similarity (DNS) and anatomy-aware contrastive learning to learn discriminative and contrast-invariance deep structural image representations (DSIR) without the need for anatomical delineations or pre-aligned training images. We evaluate our method on multiphase CT, abdomen MR-CT, and brain MR T1w-T2w registration. Comprehensive results demonstrate that our method is superior to the conventional local structural representation and statistical-based similarity measures in terms of discriminability and accuracy.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. Medical image registration in image guided surgery: Issues, challenges and research opportunities. Biocybernetics and Biomedical Engineering, 38(1):71–89, 2018.
  2. John Ashburner. A fast diffeomorphic image registration algorithm. NeuroImage, 38(1):95–113, 2007.
  3. Advanced normalization tools (ANTS). Insight j, 2(365):1–35, 2009.
  4. A reproducible evaluation of ants similarity metric performance in brain image registration. NeuroImage, 54(3):2033–2044, 2011.
  5. Matching in the wild: Learning anatomical embeddings for multi-modality images. CoRR, abs/2307.03535, 2023.
  6. Voxelmorph: A learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging, 38(8):1788–1800, 2019.
  7. Unsupervised 3d registration through optimization-guided cyclical self-training. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 677–687. Springer, 2023.
  8. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020.
  9. Contrareg: Contrastive learning of multi-modality unsupervised deformable image registration. In Medical Image Computing and Computer Assisted Intervention, pages 66–77, 2022.
  10. Matthias Eisenmann et al. Biomedical image analysis competitions: The state of current participation practice. arXiv preprint arXiv:2212.08568, 2022.
  11. Learning iterative optimisation for deformable image registration of lung ct with recurrent convolutional networks. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VI, pages 301–309. Springer, 2022.
  12. Automated learning for deformable medical image registration by jointly optimizing network architectures and objective functions. IEEE Transactions on Image Processing, 32:4880–4892, 2023.
  13. Training batchnorm and only batchnorm: On the expressive power of random features in cnns. arXiv preprint arXiv:2003.00152, 2020.
  14. Guiding multimodal registration with learned optimization updates. Medical image analysis, 41:2–17, 2017.
  15. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020a.
  16. Momentum contrast for unsupervised visual representation learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9726–9735, 2020b.
  17. MIND: modality independent neighbourhood descriptor for multi-modal deformable registration. Medical Image Analysis, 16(7):1423–1435, 2012a.
  18. Globally optimal deformable registration on a minimum spanning tree using dense displacement sampling. In Medical Image Computing and Computer Assisted Intervention, pages 115–122, 2012b.
  19. Globally optimal deformable registration on a minimum spanning tree using dense displacement sampling. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2012: 15th International Conference, Nice, France, October 1-5, 2012, Proceedings, Part III 15, pages 115–122. Springer, 2012c.
  20. Towards realtime multimodal fusion for image-guided interventions using self-similarities. In Medical Image Computing and Computer-Assisted Intervention, pages 187–194, 2013.
  21. Olivier Henaff. Data-efficient image recognition with contrastive predictive coding. In International conference on machine learning, pages 4182–4192. PMLR, 2020.
  22. Learn2reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning. IEEE Transactions on Medical Imaging, 2022.
  23. Hypermorph: Amortized hyperparameter learning for image registration. In International Conference on Information Processing in Medical Imaging, 2021.
  24. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  25. Learning similarity measure for multi-modal 3d image registration. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 186–193. IEEE, 2009.
  26. Samconvex: Fast discrete optimization for CT registration using self-supervised anatomical embedding and correlation pyramid. In Medical Image Computing and Computer Assisted Intervention, pages 559–569, 2023.
  27. JSSR: A joint synthesis, segmentation, and registration system for 3d multi-modal image alignment of large-scale pathological CT scans. In Computer Vision - ECCV 2020 - 16th European Conference, pages 257–274, 2020a.
  28. Bi-level probabilistic feature learning for deformable image registration. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI, pages 723–730. ijcai.org, 2020b.
  29. Learning deformable image registration from optimization: Perspective, modules, bilevel training and beyond. IEEE Transactions on Pattern Analysis Machine Intelligence, 44(11):7688–7704, 2022.
  30. An integrated approach to segmentation and nonrigid registration for application in image-guided pelvic radiotherapy. Medical Image Analysis, 15(5):772–785, 2011.
  31. Rectifier nonlinearities improve neural network acoustic models. In Proc. icml, page 3. Atlanta, GA, 2013.
  32. Multimodality image registration by maximization of mutual information. IEEE transactions on Medical Imaging, 16(2):187–198, 1997.
  33. A review of medical image registration. Interactive image-guided neurosurgery, 1:17–44, 1993.
  34. The multimodal brain tumor image segmentation benchmark (brats). IEEE Transactions on Medical Imaging, 34(10):1993–2024, 2015.
  35. Fast symmetric diffeomorphic image registration with convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4644–4653, 2020a.
  36. Affine medical image registration with coarse-to-fine vision transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20835–20844, 2022a.
  37. Conditional deformable image registration with convolutional neural network. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, pages 35–45. Springer, 2021.
  38. Robust image registration with absent correspondences in pre-operative and follow-up brain mri scans of diffuse glioma patients. In International MICCAI Brainlesion Workshop, pages 231–240. Springer, 2022b.
  39. Unsupervised deformable image registration with absent correspondences in pre-operative and post-recurrence brain tumor mri scans. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 25–35. Springer, 2022c.
  40. Deformable medical image registration under distribution shifts with neural instance optimization. In International Workshop on Machine Learning in Medical Imaging, pages 126–136. Springer, 2023.
  41. Tony C. W. Mok and Albert C. S. Chung. Large deformation diffeomorphic image registration with laplacian pyramid networks. In Medical Image Computing and Computer Assisted Intervention, pages 211–221, 2020b.
  42. Deriving anatomical context from 4d ultrasound. In Vcbm, pages 173–180, 2014.
  43. Contrastive learning for unpaired image-to-image translation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16, pages 319–345. Springer, 2020.
  44. Comir: Contrastive multimodal image representation for registration. Advances in neural information processing systems, 33:18433–18444, 2020.
  45. Image registration by maximization of combined mutual information and gradient information. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2000: Third International Conference, Pittsburgh, PA, USA, October 11-14, 2000. Proceedings 3, pages 452–461. Springer, 2000.
  46. Michael JD Powell et al. The bobyqa algorithm for bound constrained optimization without derivatives. Cambridge NA Report NA2009/06, University of Cambridge, Cambridge, 26, 2009.
  47. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234–241. Springer, 2015.
  48. Matching local self-similarities across images and videos. In 2007 IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8. IEEE, 2007.
  49. Hammer: hierarchical attribute matching mechanism for elastic registration. IEEE Transactions on medical imaging, 21(11):1421–1439, 2002.
  50. Medical image registration based on uncoupled learning and accumulative enhancement. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV, pages 3–13. Springer, 2021.
  51. Fast 3d registration with accurate optimisation and little learning for learn2reg 2021. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 174–179. Springer, 2021.
  52. Free-form deformation using lower-order b-spline for nonrigid image registration. In Medical Image Computing and Computer Assisted Intervention, pages 194–201, 2014.
  53. Multi-site infant brain segmentation algorithms: The iseg-2019 challenge. IEEE Trans. Medical Imaging, 40(5):1363–1376, 2021.
  54. Representation learning with contrastive predictive coding. CoRR, abs/1807.03748, 2018.
  55. Alignment by maximization of mutual information. International journal of computer vision, 24(2):137–154, 1997.
  56. Automatic ct-ultrasound registration for diagnostic imaging and image-guided intervention. Medical image analysis, 12(5):577–585, 2008.
  57. Evaluation of mri to ultrasound registration methods for brain shift correction: the curious2018 challenge. IEEE transactions on medical imaging, 39(3):777–786, 2019.
  58. SAM: self-supervised learning of pixel-wise anatomical embeddings in radiological images. IEEE Trans. Medical Imaging, 41(10):2658–2669, 2022.
  59. Modified bézier curves with shape-preserving characteristics using differential evolution optimization algorithm. Advances in Numerical Analysis, 2013:1–8, 2013.
  60. Learning-based us-mr liver image registration with spatial priors. In International conference on medical image computing and computer-assisted intervention, pages 174–184. Springer, 2022.
  61. Richard Zhang. Making convolutional networks shift-invariant again. In International conference on machine learning, pages 7324–7334. PMLR, 2019.
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now